Student Learning Outcomes and Assessment Practices
Assessment Principles
While the T&L is not a credit-bearing program with full-time equivalent students (FTES) the department does have shared student learning outcomes (SLO’s) for various aspects of our activities. All T&L library faculty participate in the first year experience curriculum and provide general reference/research help. These areas most clearly lend themselves to shared learning outcomes.
In addition, subject specialist library faculty are responsible for partnering with particular academic programs to provide students with learning experiences focused on inquiry, research, and curiosity. There is more variability here in terms of learning outcomes. Subject specialists can use the departments’ shared SLO’s as a framework for their subject-oriented work.
Assessing shared SLO’s aid the department in planning, decision-making, and communication, within and beyond the Library organization about how T&L contributes to student learning at CSUSM. It will allow us to change and/or update our efforts to be as effective as possible.
To this end, the group agreed that assessment efforts in T&L will be iterative, defined in scope, and manageable in size. We commit to assessment efforts that:
- focus on and flow from clear and important goals, especially those to improve curriculum and pedagogy but also regarding planning, budgeting, and accountability;
- are supported with appropriate resources, including time, expertise, guidance, financial resources, support and feedback;
- seek to involve the active participation of appropriate stakeholders in decisions stemming from the results;
- are communicated widely and transparently.
These principles were considered and adopted in Fall 2014. These are based heavily on Suskie’s Principles of Assessment.[1]
Learning Outcomes
Drafting learning outcomes that are applicable to all aspects of T&L is especially challenging since the curriculum exists out of the common boundaries of credit-bearing, degree granting programs. However, T&L’s focus on developing the student-scholar as both an information consumer and knowledge producer indicates that the primary focus is not merely the development of research skills or information literacy. Instead, research skills and information literacy are the manifestation of the habits of mind, dispositions, and approaches of a student-scholar. With this in mind, T&L’s learning outcomes are divided into two categories: the development of the student scholar (and beyond the academy, a lifelong learner) and the skills and abilities needed to use and create information.
Becoming a student scholar/engaged community member
As a result of the learning experiences from T&L faculty, CSUSM graduates will have the dispositions and habits of mind of engaged members of their regional and global community. The will:
- demonstrate engagement in the scholarly process;
- display characteristics of a student-scholar identity;
- describe actions of lifelong learners.
| Above Standard 4 | Standard 3 | Sufficient 2 | Emerging 1 | |
| Engagement in the scholarly process | Pursues scholarly inquiry as an engaged actor in the scholarly process | Participates in the scholarly inquiry process | Articulates the variations of the scholarly inquiry process | Describes common steps in scholarly inquiry |
| Student-scholar identity | Values role as participant in knowledge creation as student-scholar | Participates in knowledge creation as student scholar | Articulates the characteristics of student-scholar.
May demonstrate characteristics of knowledge creator (e.g. provides a conclusion different from other scholars). |
Awareness of student-scholar as user and creator of knowledge |
| Dispositions and habits of mind as student scholar | Consistently approaches inquiry as a student scholar.
Critical user of information as well as knowledge creator |
Inconsistently demonstrates a variety of characteristics of student-scholars | Describes a variety of dispositions and habits of student-scholars
May demonstrate some characteristics of knowledge creator |
Awareness of disposition and habits of student-scholars
(e.g. research does not happen in one sitting) |
Information use and creation
As a result of the learning experiences from T&L faculty, CSUSM graduates will:
- Identify: Identify issues, perspectives, or problems clearly and in multiple contexts;
- Evaluate: Distinguish between relevant/credible sources, ideas, and information;
- Analyze: Consider multiple points of view, perspectives, or solutions with a comparative approach;
- Synthesize: Formulate clear and well-stated plans, solutions, or outcomes;
- Concludes: Provide logical conclusions that reflect full range of information from multiple and suitable perspectives;
- Act ethically: Use information in an ethical manner.
| 1
Emerging |
2
Approaching Standard |
3
Meets Standard |
4
Above Standard |
|
| Explanation of Issues
(Identify/Evaluate) |
Limited ability to distinguish fact from opinion; limited understanding of credible/relevant sources/ideas; fails to recognize multiple perspectives | Recognizes fact and opinion as categories; awareness of standards of credibility in sources/ideas; awareness of multiple perspectives | Distinguishes between fact and opinion; evaluates sources/ideas as credible or relevant; recognizes multiple perspectives | Clear method of distinguishing between fact vs opinion and credible/relevant; sophisticated understanding of multiple perspectives |
| Reasoning/
Planning (Analyze) |
Articulates possible arguments, plans, or solutions with little or no logical approach | Articulates possible arguments, plans, or solutions with imprecise reasoning | Articulates logical arguments, plans, or solutions using inductive and/or deductive reasoning skills and clear method(s) | Articulates sophisticated arguments, plans, or solutions using inductive and/or deductive reasoning skills and clear method(s) |
| Outcome
(Synthesize) |
Provides conclusions/finished ideas that are not tied to evidence; limited range of evidence/sources; demonstrates fallacious reasoning | Provides conclusions/finished ideas that are loosely tied to evidence from sources of varying credibility; demonstrates some fallacious reasoning | Provides logical conclusions/finished ideas informed by a full range of evidence from multiple and suitable sources; avoids fallacious reasoning | Provides logical conclusions/finished ideas informed by a full range of evidence from multiple and suitable sources; reasoning reflects complexity and avoids fallacies |
| Ethical Use of Information | Provides minimal attribution of information sources | Provides attribution of some information sources in some instances | Appropriately attributes information sources, but with some inconsistencies | Provides appropriate attribution of information |
CSUSM Teaching and Learning (T&L)
Assessment plan for Fall 15 – Spring 16
Introduction
The purpose of this document is to describe the planned and ongoing assessment activities in T&L for AY 2015-16. It is important to note that this is the first year that T&L has embarked on these activities. The group felt strongly about two principles: 1. that we must experiment with different approaches and strategies in order to determine what has the greatest impact and 2. implement an assessment approach that is sustainable. T&L is also in a transitional period and growing rapidly, making it even more imperative to identify an assessment approach that works for the unit not only in the present, but as the unit changes and grows.
The faculty in T&L had a series of discussions beginning in Fall 2014 about how to start conducting program-level student learning assessment. T&L faculty first articulated the program’s principles, assessment principles, and ultimately, rubrics to assess to categories of student learning outcomes (SLO’s).
While not finalized at the time of these conversations, CSUSM’s Undergraduate Learning Outcome 2 aligns with the aforementioned SLO’s (language specific to information literacy learning outcomes are in italics) :
2) Comprehensive and critical thinkers. Students will be able to:
- Identify key concepts and develop a foundation for future inquiry
- Analyze complex problems and develop solutions by applying quantitative and qualitative reasoning, integrating knowledge and skills from a variety of disciplines
- Construct well-reasoned arguments based on evidence
T&L faculty decided to begin programmatic assessment within the GEL 101 Research Module (RM). All T&L faculty teach the Research Module using a shared curriculum. For this reason, conducting this first assessment cycle within the Research Module could generate findings that would impact all librarians.
Pre- and post-test assessment
As with any assessment, T&L faculty then identified which of the T&L SLO’s to assess. The group sought to identify not only those SLO’s they were curious about, but which could have some impact on how to ensure students achieve that outcome. The group decided to assess the SLO “Identify/Evaluate”, as noted in the chart below. Language in bold is the specific aspect of the learning outcome that the group chose to focus on.
| 1
Emerging |
2
Approaching Standard |
3
Meets Standard |
4
Above Standard |
|
| Explanation of Issues
(Identify/ Evaluate) |
Limited ability to distinguish fact from opinion; limited understanding of credible/relevant sources/ideas; fails to recognize multiple perspectives | Recognizes fact and opinion as categories; awareness of standards of credibility in sources/idea; awareness of multiple perspectives | Distinguishes between fact and opinion; evaluates sources/ideas as credible or relevant; recognizes multiple perspectives | Clear method of distinguishing between fact vs opinion and credible/relevant; sophisticated understanding of multiple perspectives |
The language in bold of the above learning outcome is often taught to students as recognizing the characteristics of scholarly articles as opposed to information resources from non-scholarly sources. Within a scholarly community such as a university, the scholarly, peer-reviewed article in a research journal is widely considered to be a highly desirable and credible type of information source. Students’ knowledge of credible information resources is essential to being able to identify academic journal articles.
To this end, for Fall 2015, the T&L faculty will be assessing students’ ability to determine if an information source is a scholarly journal article or another type of information source that be helpful in defining terms, describing concepts, or providing context to the topic of the student’s inquiry. It most commonly is found in the form of web sites found on open web, Wikipedia entries, or other sources whose audience is not experts in the particular field. Students completing the RM are expected to recognize information sources that are different from scholarly sources. Scholarly sources are produced by experts and are produced for other experts in a field. They are distributed most typically through scholarly journals and are sometimes found in special search engines/databases.
Research Module assessment survey
T&L decided to create a separate assessment tool for this selected learning outcome for two main reasons: 1. determining if a student achieved the outcome could be done fairly easily and 2. a desire to automate the data collection and analysis because of the significant numbers of first year students taking GEL.
The group also determined that first year students could be expected to achieve a level of “Approaching Standard” on this outcome at the conclusion of the RM. The assessment would be administered at the beginning and end of the RM. Students enrolled in GEL may also be taking GEW 101 and/or GEO 102. Because students work with a librarian in these courses as well, the assessment tool will include items to determine if students are enrolled or have completed the course. There may be a correlation between students’ performance on the assessment tool and co-enrollment in these courses.
With these parameters, the assessment tool asked:
- If students were currently enrolled in GEW, GEO, or both courses.
- When provided with two links to a web page, if students could determine whether the page was a scholarly source.
- Students to rate their confidence in their determination about the information source they were linked to.
The assessment tool will be administered via CougarCourses to students enrolled in GEL 101 for Fall 2015. The tool will be administered with the homework that is required to be completed immediately before the Research Module. The post-test will be administered at the session after the conclusion of the RM or during the last RM session. Institutional Planning and Assessment (IPA) will be assisting by administering the survey using SurveyGizmo and providing basic data analysis. Results are expected in December 2015-January 2016, and a report will be drafted for T&L’s review.
Analysis of bibliographies in infographics
In addition to the assessment described above, T&L faculty decided to analyze the bibliographies of infographics from students in the RM. Since the infographics could be considered a signature assignment of the RM, the bibliographies could provide additional information about students’ abilities to identify credible information sources within the scholarly community.
Library faculty often require different numbers of scholarly articles in an infographic’s bibliography. Some require 1-2 and some require more. During Summer 2015, faculty assessed the bibliographies of 281 infographics by one criterion: Does the bibliography include the required number of scholarly sources? Bibliographies were assessed on a 3 point scale: No scholarly sources, Some (any amount from one to one less than the required amount), All (required number of sources included).
This data will be discussed by T&L faculty in Spring 2016, along with the data from the RM assessment that is being administered in Fall 2015.
Student learning outcomes for information literacy in the first year
Lastly, the T&L Director, in collaboration with the recently created First Year Experience Librarian, and course coordinators for GEW 101 (General Education: Writing) and GEO 102 (General Education: Oral Communication) are working to articulate student learning outcomes and an assessment approach for information literacy in the first year. This goes beyond the scope of the GEL RM and includes the information literacy components within all three courses.
Next steps
There are several ongoing efforts within T&L related to assessment during AY 2015-16. This indicates significant interest in this aspect of teaching and learning. A concern, however, is that these efforts are not connected into a single framework or approach. To that end, T&L will be engaging in a review of the assessment efforts for this year at the conclusion of AY 2015-16 to:
- Identify how the multiple “sets” of student learning outcomes are connected. That is, there are two categories of learning outcomes for T&L as a whole as well as a separate list of SLO’s for first year information literacy.
- Consider an assessment approach that is sustainable.
- Identify assessment practices from this year that were particularly useful.
[1] http://planning.iupui.edu/page/download/?key=129728430&path=/consult/conferences/national/06/handouts/monday/suskie.pdf