Course Assessment - The CIEQ Instrument

CIEQ is an assessment instrument that has been demonstrated to provide reliable data regarding how students feel about a course. It only measures student affective response to the course and the instructor, it does not measure learning or success at achieving educational outcomes. Student learning outcomes should be measured and reported as part of the degree program’s assessment (TK-20 Assessment) and the instructor’s self assessment of their teaching. Assessment of student learning outcomes can provide quantitative and/or qualitative data on what is working, inform the instructor on strategies for improving the course through changes in teaching methodologies, course materials, student learning assessment instruments and other areas, and document student achievement of the learning outcomes. 

The CIEQ score for an instructor is influenced by a number of factors and comparison of scores among faculty must be done with this in mind. The most appropriate use of CIEQ scores by an administrator is to look for trends over time or significant changes in evaluation scores from one year to the next. CIEQ scores should not play a major role in promotion or tenure decisions as the sole indicator of instructional competence.

Student evaluation scores are known to be affected by the following factors:

• Scores are impacted by disciplines, with the life, physical, chemical and earth sciences and mathematics receiving the lowest scores and arts and humanities the highest scores.
• Scores are impacted by class level, with first year classes receiving lowest scores and graduate courses the highest scores.
• New faculty tend to have lower scores that increase with more teaching experience.
•  In laboratory classes your score may be markedly affected by the performance of your teaching/learning assistant.
• Departments that teach service courses for non-majors often receive low CIEQ scores.

 CASNR is now calculating a composite CIEQ percentile that adjusts for class level and department as an index of student satisfaction. This is listed as ADJ PERC on the report form. Faculty receiving scores <25% should reflect on whether the class environment is having a negative impact on learning. Variation of >10% may not be significant.

Administration of the CIEQ Instrument:

• Beginning with the Fall 2016 semester, the CIEQ Assessment Instrument will be administered electronically for all CASNR courses.
• The CIEQ will be available to students the last few weeks of the semester, and students will be informed of the availability of the instrument via an email sent from the UNL online evaluation system (crseval.unl.edu). The email will contain a link that will take students directly to the evaluation.
• To encourage students to complete the evaluation, weekly reminder emails will be sent to students who have not filled out the CIEQ.
• The evaluation will close on the Sunday following Finals Week.
• Instructors will be able to monitor response rates, but will not be able to see which students have completed the evaluation.
• For courses with more than one instructor, the system has been configured so that a separate evaluation will be completed for each instructor listed in MyRED. In this case, the evaluation results for each instructor will be accessible only to him/her. If you would prefer that a “course”-level evaluation be administered, contact your unit’s CIEQ coordinator and he/she can set that up. If you choose this option, the results will be accessible to all instructors identified in MyRED.
• Your unit CIEQ coordinator can also remove the evaluation from courses for which an evaluation does not make sense (for example, thesis hours).
• To encourage high response rates, the CASNR FAC is endorsing the following methods to encourage student participation in the course evaluation process:
• As with paper evaluations, emphasize to students that the online evaluation is anonymous and faculty will not receive responses until after grades are submitted for the semester.
• In the syllabus, inform students of their responsibility to the course to fill out the evaluation at the end of the semester.
• Most importantly, similar to the administration of the paper-based CIEQ, continue to reserve time at the end of class for students to fill out the evaluations in the classroom.  Setting aside class time is key to a larger response rate. Students will need to bring a web-enabled mobile device such as a laptop, tablet, or smartphone.  The instructor should announce this in advance of the class session, and put the announcement in the syllabus or on Canvas.  As with paper delivery, the instructor leaves the room during this time.  Any student who completed the web form ahead of class is allowed to leave when the instructor does. 
• Consider using social pressure to complete the evaluation. For example, if a significant percentage of students complete the evaluation, a bonus of some kind is awarded. Another example of social pressure would be to multiply the response rate by a small constant (say, 2) and that number will be added to everyone’s score before letter grades are determined. The more people that participate, the higher everyone’s reward.
• Additionally, consider flexible incentives as an alternative to offering bonus points. For example, since course evaluation is an expectation of a student’s participation in a course, students could be asked to print a copy of the confirmation page of the evaluation and submit it as a (required) homework assignment.
• Consider reminding students frequently to complete the course evaluation.

Accessing Results of the CIEQ

•  After the evaluation closes and final grades are filed, instructors can log onto crseval.unl.edu to view the raw data and student comments. 
• Roughly six weeks after the end of the semester, CASNR will provide the designated unit CIEQ Coordinator with the instructor/course summary report that includes the composite CIEQ percentiles that adjust for class level and department/unit. It is the responsibility of the CIEQ Coordinator to forward the instructor/course summary report to the instructor.  CASNR also will provide the unit CIEQ Coordinator with a cop of the unit summary report.
• The instructor has the option to share the written comments from students with their unit administrators or include as part of their annual evaluation materials or promotion and tenure packet.

Interpretation of CIEQ Results

• Check that the sample size is at least half the course enrollment.
•  Under Subscale results, check that the Reliability (REL) is high. Values <0.65 are suspect. Check decile results of CIEQ data normalized for:
•  INSTR RANK: the rank of the instructor
•  CLASS LEVEL: all courses at the same level (e.g. Junior, Senior, Graduate, etc.)
•  CASNR: All courses within CASNR
•  DEPT: All courses in that UNL Department
• CASNR is now calculating a composite of the CLASS LEVEL and DEPT percentile as an index of student satisfaction. This is the Adjusted Percentile (ADJ PERC)

Interpretation of Results Groupings

• 0 to 25%: Substantial improvement needed
• 25-70%: Some improvement possible
• 70-99%: Student responses indicate course is satisfactory

 Within a decile class, movement up or down by one number (10%) is not significant.
Deciles and percentiles report student satisfaction. More analysis and collaboration with the instructor are likely necessary to understand and implement appropriate changes in the course.