CDL’s Collection Development and Management: Licensed Resources program has developed a value-based strategy which is now used as a major part of the University of California’s journal collection planning process.  The strategy involves using objective metrics to calculate the value of scholarly journals and identify titles that make a greater or lesser contribution to the University’s mission of teaching, research, and public service. The value-based process is objective and quantifiable and is based on measures of utility, quality, and cost effectiveness, with a goal of alignment to UC’s user communities and programmatic needs.

A key aspect of this strategy is the use of a Weighted Value Algorithm by Subject Category to assess multiple vectors of value for each journal title under review.  This methodology compares each UC e-journal title licensed for systemwide use at the University of California against other UC-licensed titles within the same subject category according to a variety of objective value indicators, in order to arrive at a comparative value for each journal within the UC shared licensing portfolio. The analysis covers over 8,600 journals in 36 UC licensed e-journal packages.

The algorithm takes into account three vectors of value encompassing six data metrics: Utility (usage and citations), Quality (Impact Factor;1 SNIP 2) and Cost Effectiveness (cost per use; cost per SNIP). To establish a baseline for comparison, median values are calculated for each of these metrics within 158 different subject categories. After analyzing a variety of available subject schemes, the Hierarchical Interface to LC Classification (HILCC) 3 developed at Columbia University was selected as the best subject system for this purpose.

The table below demonstrates the differences in median values across all UC licensed titles for two sample subjects under each of the following broad subject categories: Life and Health Sciences, Physical Science and Engineering, Social Sciences, and Arts and Humanities.

A numerical score is assigned to each title for each of these measures depending on whether the value for each metric is above or below the median for that subject. Besides median values, other benchmark values are also established for some metrics to allow for further differentiation. For usage, titles in the lowest quartile of usage in a given subject category receive the lowest possible usage score. The value categories and the CDL algorithm by which they are assigned are shown below.

Finally, each title is assigned an overall value of High, Medium, Low, or Very Low based on its total score that combines the three measurement categories. The Utility category, which includes usage, is given the most weight in determining the overall score.

It is important to note that the value categories assigned using this algorithm are not meant to be absolute indicators of journal quality, but instead are an attempt to apply a set of consistent and rigorous criteria by which to analyze relative value within a specific institutional context. The citation, usage and cost data are all institution-specific measures, as are the median values used by the algorithm. This methodology answers the question “How much value does our institution derive from Journal X compared to other journals that we license in the same discipline?” where ‘value’ is defined as a combined measure of quality, utility, and cost-effectiveness.

In addition to the CDL Weighted Value Algorithm, many other metrics are compiled, calculated, and provided to UC campus librarians by CDL to ensure the richest possible set of information with which to make important journal collection decisions. As the quantity and variety of useful metrics has grown, CDL has found it necessary to develop a database to manage and calculate all this information for the over 8,600 journals that are licensed systemwide. The database was initially populated with a list of UC licensed journals downloaded from CDL’s electronic resource management system; additional data were then imported from numerous sources, including vendor usage statistics, CDL cost information, Impact Factors, UC citation rates, and other sources. The database associates the data for each title and calculates the subject and title-level metrics and score assignments for use in further analysis.

For additional information contact Jacqueline Wilson, Senior Associate for Collection Development and Management, California Digital Library (jacqueline.wilson@ucop.edu) or Chan Li, Senior Data Analyst, California Digital Library (chan.li@ucop.edu)

Foot Notes:

University of California Libraries Academic eBook Reader Survey  – Monday, October 11 through Monday, October 25, 2010

The UC Libraries charged the Springer eBook Task Force to create a survey to collect the opinions of UC faculty, staff, and students about their preferences for print, ebooks, etextbooks and their experiences using the Springer eBook pilot collection. Information on the Springer eBook pilot can be found at:

http://www.cdlib.org/services/collections/springerebooks/

UC Library employees are encouraged to  participate in the forthcoming UC Libraries Academic eBook survey. Please visit the short survey when it becomes available and let us know about your preferences and experiences with ebooks in general and the Springer eBook pilot. The survey will be available  from Monday,  October 11, 2010 through Monday, October 25, 2010.

Springer eBook Task Force members and each campus Collection Development Officer are coordinating local outreach for the survey. We hope to get as many responses as possible in order to develop a clear picture of UC reader preferences. The UC Collection Development Officers will use the results of the survey to make decisions about future Springer eBook purchases as well as decisions about systemwide licensing of other ebook packages.

Links to the survey will be available on each campus library web site and on the SpringerLink site.  All respondents, including library staff, who choose to provide their name and email address will be entered into a drawing for one of five $50 gift certificates to the UC bookstore of their choice.  Individual responses to the survey will be kept confidential.  Survey results will be available later this fall.

The survey was developed by Brian Quigley (Berkeley), Felicia Poe (CDL), Michele Potter (Riverside), Chan Li (CDL), and Jacqueline Wilson (CDL).

For more information contact:  Jacqueline.Wilson@ucop.edu

CDL will host four sessions of Value-based Journal Metrics/Journal Review Process, 2011  on:

Wednesday, May 12, 2:00 – 3:30 PM
Thursday, May 13, 3:00 – 4:30 PM
Monday, May 17, 11:00 AM – 12:30 PM
Tuesday, May 18, 9:00 AM – 10:30 AM

The webinars will cover:

• Value-based metrics
• Value-based metrics algorithm revisions
• Definitions of terms used in the journal review process
• Title voting options
• Usage data: sources, validity, normalization, ScholarlyStats, COUNTER 3
• Review of Springer journals spreadsheet
• 2011 Journal Title Review Process overview and timeline

To enroll in a webinar go to:  http://www.cdlib.org/services/collections/protected/survey/index.html [password protected]; UC users can request a password here.

 If you have questions about the webinars, please contact Chan Li at Chan.Li@ucop.edu

In 2008, the UC libraries adopted a new value-based strategy developed by CDL which is now used in negotiations with major publishers.  The strategy involves using objective metrics to compare the value of scholarly journals, establish price containment targets, and identify titles of poor value to consider for potential cancellation. The process is objective and quantifiable and is based both on cost and value with a goal of leading to more sustainable prices overall.   

A key aspect of this new methodology is the use of a Weighted Value Algorithm to assess multiple vectors of value for each journal title under review.  Value is assessed in three overall categories:  Utility, Quality, and Cost Effectiveness.  For example, usage statistics contribute to a journal’s Utility score, impact factor contributes to its Quality score, while both cost per use and cost per impact factor contribute to its Cost Effectiveness score.  A composite score is then assigned to each journal to assess its overall value in comparison to other journals in the same broad subject category.  In addition to the weighted value algorithm, many other metrics are compiled and provided to campus librarians by CDL to ensure the richest possible set of information with which to make important selection decisions.

In order to assess the usefulness of the value metrics used in the 2010 journal title review and identify ways to improve the process for 2011, CDL made an online survey available to UC Collection Development Officers, campus bibliographers, and other librarians who participated in last year’s journal title review. CDL staff members Chan Li and Jacqueline Wilson developed the survey and analyzed the results.  Sixty-two librarians on nine campuses completed the survey. The majority of respondents indicated they were bibliographers.  Participants subject specialties included:  45% social science & humanities; 29% science & technology; 18% life & health sciences; and 13 % other. The survey instrument and full survey results can be found at:  http://www.cdlib.org/services/collections/protected/survey/index.html [password protected]; UC users can request a password here.

The majority of survey participants found the CDL Weighted Value Metrics to be useful, but also wanted to increase the validity of the metrics.  As a result of the survey, CDL will make the following changes to the metrics for 2011:

• Restructure the weighted value algorithm to encompass additional data points and to better account for missing data elements.  Titles with little data will get low scores so they can be reviewed by the campuses.
• Explore the feasibility of adding potential ILL costs for titles which may be cancelled
• Correct usage data anomalies (normalize data & note problems)
• Add a new major subject category division: Arts and Humanities
• Seek  additional citation data for social science & arts and humanities titles
• Bundled titles will be treated as one entity
• Unsubscribed titles which were  included in packages at no cost will be shown with their publisher list price
• Eigenfactor use and value will be reconsidered as it is less widely understood and tends to track closely to impact factor

New journal metrics which were of interest to at least 50 percent of the survey participants will be investigated for possible inclusion in this year’s journal title review process.  These metrics include:  # of editors by campus, # of authors by campus, average UC use per article, average UC cost per article, 5 year usage trends, and 3 year usage trends.

Survey participants also identified a number of general changes they would like to see in the next journal review process:

• Lengthen the time given for campuses to review titles and metrics. Journal package lists up for review will start going out by the end of April in time for campuses to coordinate systemwide decisions with local collection decisions
• Lists of new and transfer titles for journal packages that are up for review  will go out separately in the early Fall
• CDL will clarify definitions of “subscribed” & “unsubscribed,” “licensed “and “not licensed,” and  “Historical UC Subscription”
• CDL will continue to build support for niche titles into the review process
• CDL will revise voting choices with clearer definitions
• Usual campus co-shares will be used allocate costs for each journal package; title votes will not be used to re-calculate campus cost shares

 CDL will host a number of webinars for campus librarians in May to review the metrics and procedures for this year’s title review process.  More information on the webinars can be found at:  http://www.cdlib.org/services/collections/protected/survey/index.html.

 Please contact Chan Li (Chan.Li@ucop.edu) or Jacqueline Wilson (Jacqueline.Wilson@ucop.edu) if you have any questions or would like additional information.

The University of California Libraries will be evaluating Scopus <http://www.scopus.com>, a large interdisciplinary abstract and citation database of research literature and selected web sites developed by the publisher Elsevier.  The arrangement is part of the journals license for ScienceDirect negotiated by the California Digital Library on behalf of the ten campuses.  The Scopus database will be available for evaluation until February 28, 2010.  During the Scopus evaluation period members of the UC community will be invited to consider its coverage and functionalities in comparison with similar databases.  Scopus will also be evaluated to determine if its unique qualities and capabilities are of interest and strategic value to UC.

Scopus currently selectively indexes abstracts and provides citation analysis since 1996 for approximately 16,000+ peer-reviewed journals from more than 4,000 publishers as well as open access journals, published sets of conference proceedings, trade publications and selected web pages.  It provides searching tools that allow users to see an overview of desired search results and refine them to the most relevant hits.  It also allows users to create alerts of new citations and provides tools designed to support research collaboration and assess journal quality.  Citations in Scopus are linked to available full-text articles on each campus via UC eLinks.  Although most comprehensive in the sciences, Scopus is currently expanding its coverage of the social sciences and humanities literature.

The Collection Development Committee has charged a Scopus Rollout and Evaluation Task Force to publicize and to lead a formal evaluation of Scopus.  The Task Force members are:  Myra Appel (Davis), Janet Carter (Los Angeles), Rob Melton (San Diego), Barbara Schader (Riverside), Jeff Williams (San Diego), Ivy Anderson (CDL), and Jacqueline Wilson (CDL, Chair).  The Scopus Task Force is developing an informative web site which will include a model campus publicity document, model speaking points for UC training librarians, a model PowerPoint presentation, and other documents.  This web site will be announced in the next week or two.

In addition, each campus library has designed a rollout liaison who will coordinate librarian training and local campus publicity.  The Scopus rollout liaisons are:  Myra Appel Davis), Marcus Banks (San Francisco), Janet Carter (Los Angeles), Andrea Duda (Santa Barbara), Susan Mikkelsen (Merced), Barbara Schader (Riverside), John Sisson (Irvine), Beth Weil (Berkeley), Jeff Williams (San Diego), and a soon to be announced liaison for Santa Cruz.  Hands-on training sessions will be held in Northern California in April and in Southern California in May.

Please contact your local campus rollout liaison or Jacqueline Wilson (Jacqueline.Wilson@ucop.edu) Scopus Task Force Chair for more information.

A task force appointed by the Scholarly Communications Officers (SCO) worked with the California Digital Library (CDL) to prepare information on the UC/Springer Open Access Journal Publishing Pilot, originally announced in January, for distribution to UC authors on each campus.  The information can be found on the Reshaping Scholarly Communication site at http://osc.universityofcalifornia.edu/alternatives/springer_faq.html.  This site includes a FAQ, a link to the Springer journals covered by the pilot as well as a short list of titles excluded from the program.  It is expected that campuses will customize the information to suit their local audience.

As was noted in the original message about this arrangement, it will be important to the success of the pilot that as many UC authors as possible know about this open access opportunity.  Scholarly Communications Officers and others will distribute information about this agreement to appropriate faculty and other authors on their campus as part of the local publicity plans that they have developed.

Final versions of the Springer journal articles published during the pilot will be available in the eScholarship Repository beginning in the fall.

CDL is pleased to have made this ground-breaking arrangement with Springer on behalf of UC authors and the Task Force is looking forward to assessing the results of this experiment as it unfolds over the next two years.

UC/Springer Open Access Journal Publishing Pilot Task Group:

Ivy Anderson (CDL)
Catherine Mitchell (CDL)
Margaret Phillips (Berkeley)
Jacqueline Wilson (CDL, Chair)

CDL Collections and eScholarship in partnership with the UC Scholarly Communication Officers (SCOs) are piloting a ground-breaking open access publishing arrangement with Springer on behalf of UC faculty.  The two year experiment has already begun.  CDL sent out a press release about the pilot program on January 21, 2009.  The publicity was distributed to a variety of library information sources as well as the Chronicle of Higher Education and other standard UCOP media outlets.  Springer will also send the release to selected European contacts.  Information has been also posted on the CDL web site and on the Reshaping Scholarly Communication web site.

Laine Farley, CDL Executive Director, and Ivy Anderson, CDL Director, Collection Development and Management, have requested that the University Librarians communicate with appropriate faculty on their campuses to notify them of this pilot.  It will be important to the success of the pilot that as many UC faculty as possible know about this open access opportunity.  According to Springer, approximately 1,500 articles with at least one UC author are published annually in Springer journals. CDL will also be working with the SCOs on additional campus outreach as well as on an evaluation plan.

An article is eligible for Springer’s open access publishing option (called Springer Open Choice) if even one of the authors of an article is from UC. To invoke the Open Choice option, the corresponding or submitting author simply selects a UC campus affiliation from a drop-down box that appears on the acceptance screens that he or she completes once the article has been accepted for publication.  Therefore, the submitting author, if not from UC, must know about the program and indicate that one of their co-authors possesses a UC affiliation. This requires that the UC authors be familiar with the pilot and remember to inform their co-authors about the program.  A message on the acceptance screen will indicate that the Open Choice option is being made available to them at no charge through an arrangement with the University of California Libraries.

CDL will clarify some remaining issues about placing the final published Springer OA articles in the eScholarship Repository and we expect to begin this part of the pilot soon.

Please address questions to Jacqueline Wilson (jacqueline.wilson@ucop.edu), Senior Associate for Collections, California Digital Library.