Library Assessment Report for Academic Year (2014 2015): Institutional Effectiveness I. Institutional Effectiveness Goals: 1. Select, acquire and organize resources to be easily discoverable and meaningfully incorporated into the learning and research process. Collections should have a balance of formats and content reflecting current curricular needs and college priorities for research, scholarship, teaching and learning; technology should be up to date and include common learning tools. 1. Collections (print/electronic), including books, journals, media and archives. 2. Learning Technology for student use within and outside the l ibrary. 3. Interlibrary Loan to temporarily acquire and provide those items we do not own. 2. Provide quality assistance, guidance and instruction in the use of information sources, learning technologies and collections to the entire SUNY Oswego community students, faculty and staff; those on campus and off campus. 1. Research, Instruction and Technology Help, including Ask A Librarian services. 2. Special Collections/Archives 3. Access to materials 4. Customer Service 3. Provide a responsive and user friendl y online library presence, and a variety of engaging and accommodating physical spaces that meet diverse student and faculty academic needs. 1. Physical spaces 2. Website Goal # 1 Impact Two (Our Education Ecosystem) Our academic quality is demonstrated, extended, recognized, sought after and heralded. Performance Driver 1 of 6: Provide and deliver scholarly, interdisciplinary resources and innovative learning technologies that match stude nt and faculty needs, and maintain information resources that have proven essential for academic needs. Key Indicator: Support for rigorous academic programs and research. 2 Impact One (Our Students) Our students and graduates thrive through knowledge, experience, perspectives, and discovery gained here that animates and informs their work, their communities, and their personal lives. Performance Driver 1 of 4: Foster a college wide culture of caring that provides supportive, effective information literacy instruction and personalized mentoring. Key Indicator: Retention assistance, guidance and instruction in the use of information sources, learning technologies and collections
3 Impact Four (Our Institution) Our proven institutional effectiveness results in increased resources, flexibility, academic capacity and institutional success. Performance Driver 4 of 5: Put physical resources, capital assets, and technological capabilities to their highest usefulness, and ensure they are strengthened to advance continuous improvement. Key Indicator: Facilities and technology -facilities that offer appropriate and var ied spaces for individual or collaborative activities; web presence that provides seamless access to resources. II. Assessment activities completed in the recent academic year 2014 2015 a. Report goals assessed in academic year 2014 2015 : Measure Goal Results and Interpretations A. Using both vendor and local library management system reports, analyze use patterns of Purchase on Demand (POD) purchases, particularly with regard to amount and type of use. Analyze individual ebook purchase statistics for 2012 2014 from various vendo rs. (Collection Development and Electronic Resources) 1.1 In our ebook Purchase On Demand (POD) program, 182 titles have been purchased since Sept. 2012. These purchases are triggered by patron use of a select collection designed for our online programs /courses. 691 print or media POD titles have been purchased since 2010, based on interlibrary loan requests. 59% of these items have already circulated 2 or more times. POD programs remain an important component of our acquisitions process, as purchases are tailored to exactly what the patron needs. Vendor reports also indicate very high demand for subscription (not purchased) ebooks. For our two major book databases, ebrary Academic Complete & EBSCOhost ebooks, 11,127 unique titles were accessed in 28, 117 search sessions. Our collection of purchased reference titles is also well used. The Oxford collection of 26 titles, for example, had 2,225 hits this year. Given this high demand, it is important to continue offering this ebook format and multi discip linary coverage. B. Using two semesters of data collected from Aleph library management system reports, manual statistics collected on turnaways and informal student feedback, analyze use patterns for library laptop loans and unmet demand. (Access Se rvices and Technology) 1.2 When library laptops are all in circulation, pagers are checked out to students wishing to place one on during four sample weeks to record any er a laptop nor pager was available. There were no (0) turnaways.
Over the entire semester, our 30 laptops averaged 16 checkouts each per week high circulation! Pagers were checked out only 69 times all semester, with 58 of these times in the last week of class. Our conclusion is that we are almost always meeting demand and making available an appropriate number of laptops. C. Analyze Gimlet data on number, duration and content of questions received by reference librarians to inform us about the use of library faculty time for virtual and face to face reference. (Reference) 2.1 During the four weeks sampled, 1282 questions were received. The breakdown is as follows: 9% were received virtually and 91% face to face. Most questions are research based (52%), others directional (37%) and computer related (11%). Duration for most was 5 14 minutes (19%). Content themes include database searching, catalog searching, office supplies, and printing help. In the first weeks of the semester textbooks, reserve material, carrel applications, and scavenger hunts are hot topics. In the last weeks of the semester the focus shifts to citation styles and group study room availability/scheduling. This data indicates that face to face is still the preferred format for students and faculty needing research help, so a majo rity of library faculty reference service time should continue to be scheduled in face to face venues. Virtual reference has some demand and is expected to grow with institutional initiatives for online learning and other off campus programs. D. Assess Access Services student desk business and their understanding of good customer service practices. All student assistants will complete 2 3 mastery quizzes in the Fall semester; Spring hires will com plete their quizzes after a reasonable training period. (Access Services) 2.3 & 2.4 18 of 30 student assistants (60%) were given a short answer test, with an average score of 72%. Specific question analysis demonstrated that most had mastery of those tas ks handled regularly; many had low level knowledge of less typical procedures.
E. To inform the development of our collaborative service desk model, understand the types of questions that are being asked and types of services being requested through: 1) student focus groups; and 2) feedback from various service partners (CTS, OLS). (Renovation Committee) 2.1 & 3.1 1) Conducted 3 focus groups (16 students). Themes that emerged included library as space and learning support hub, furnishings, and technology support. 2) Held meetings with CTS and OLS collaborators to discuss possib le new service models and space design. It was determined that preference was for separate but adjacent service points. F. Assess the effectiveness of the newly designed library homepage by using web analytics software Crazy Egg, Google Analytics, conducting librarian surveys, and conducting first click or think aloud usability testing with students. (Webmaster) 3.2 C razy Egg and Google Analytics analysis provided solid evidence of increased use following the launch of our new library homepage. Homepage views were up 8%. Nearly 45% of clicks on the nstrating the popularity of this tool. Heat maps created by Crazy Egg suggested possible improvements to the page. Librarian surveys showed positive response to the new page, and yielded several suggestions for future usability tests. 45 students partic ipated in usability first click testing; 40 students in surveys. Most students succeeded in choosing the correct link for the assigned task, demonstrating good understanding of the design and content of the library homepage. b. Report ALL changes implemented or proposed based on assessment data gathered: (letters correspond to the relevant measure in the table above in part a.) A. Broadened the subject content of our available POD titles, as requests declined this year. B. No c hanges needed C. No changes needed D. Student assistants were instructed in their areas of weakness. Improved training, and similar testing with a greater percentage of our student workers, is planned for next year. E. Recommendations are being addres sed in designing renovations to the library lobby, underway at this time. For example, additional signage, more electrical outlets and wireless, and added scanner/printer.
F. Website improvements were made and planned based on data collected, including wording improvements for some links and added links at known points of confusion. Librarian comments were used to inform further student usability testing. c. Required resources to implement the above changes: (letters correspond to the relevant me asure in the table above in part a.) A. None B. N/A C. N/A D. Staff and librarian time for further training and assessment. E. Institutional or SUNY funding for facilities improvement. F. Time of the library webmaster and campus web developer. d. Recommended changes to the assessment process: A. Work with vendors to better define use statistics they provide. B. None C. For Gimlet data analysis, sample weeks other than the first and last of the semester, and during multiple semesters, for a more accurate picture of the diversity of question content. D. For student assistant assessment, revise test content and administer to all student assistants in this unit. E. None F. Broaden assessment to entire library website. III. A ssessment activities planned for upcoming academic year 2015 2016: Measure Goal Using vendor statistics and EBSCO A to for needed coverage of academic disciplines. (CD) 1.1 Using a survey tested in 2014 15, better understand Metro Center student preferences for library resources and services. (Instruction) 1.1 & 1.3 Using data collected via the Gimlet reference statistics program, determine the most appropriate schedule for library faculty staffing of the Research Help Desk. This will involve identifying service times with slower traffic and analyzing the type and content of questions received during t hese times. (Reference) 2.1 Collect and analyze summer data to inform discussion about library open hours and our research help schedule. Specific measures will include traffic count, materials check out, computer logins and a questions log for the Acce ss Services desk. (Access Services/Reference) 2.4 Using web analytics software and usability testing with students, broaden our assessment to the entire library website. (Webmaster) 3.2 See also: Library Assessment Report Information Literacy Learning Outcomes Submitted by Barbara Shaffer, Library Director 07/10/15