Image courtesy of the Bancroft Library

By early spring, I had a manual process in place, and in July, we went live with the automated charting, showing the trends on 11 measures from our 5 programs. The measures fall into two categories: numbers of objects and usage volume. We can add new measures easily, although we are careful about adding too many. After all, a dashboard can only get so large before the driver is too distracted to see the road ahead.

I mentioned in a July post that the tool we use is AmCharts, a tool that can extract data from .csv or .xml files and then display the data in configurable Flash charts, graphs and tables. My partner in this implementation has been Seán O’Hara of CDL’s Infrastructure and Applications Support Group. In addition to developing the dashboard reports, Seán also created an interface for the data suppliers to use to upload their monthly statistics.  He’s done great work,and I’m so grateful for his help!

I also want to thank the data suppliers, who month after month, provide the data to create the reports. This is a sizable group: Stephen Abrams, Sherri Berger, Jason Colman, Bobbi Perkins-Hill, Lisa Schiff, and Leslie Wolf. Most, if not all, of these folks have developed metrics and even dashboards at the program level. We are all learning from each others’ experiments and experiences with performance measures.

Have you tried any experiments along these lines?

In a recent post we looked at ways to keep a focus on what’s important today. In this post, we explain how to use these Key Performance Indicators to keep a focus on what’s important long term.

What are Key Performance Indicators?

Key Performance Indicators (KPI’s) are a small number of agreed-upon measurements that reflect your organization’s critical goals for success — a numerical snapshot. They are measurable, objective, and actionable. You may have heard the term “metrics” and wondered if it’s the same thing. Think of it this way: you have dozens of metrics that let you know that things are running fine on a daily basis. With KPI’s, you elevate a few of your most important metrics to become strategic touchstones for your team or service — Key Performance Indicators.

You might think KPI’s are only for managing a process or service, but you can measure anything your organization cares about: percentage of new ideas that are turned into innovations; number of software updates delivered on a timely basis; or revenue per patent.

Here are a few examples of KPI’s and some questions to show what we mean by actionable:

• Cost of service measures overall value: What is the cost of the service you are providing? A simple calculation is to total the cost of staffing and infrastructure, then divide that by the number of transactions. What is your cost per item, service or product? Does the cost match the value you are providing?
• Percentage of technical support tickets closed within a specific time measures timeliness: Are your new procedures allowing you to close tickets for your service more quickly than before? Will that reduce your cost of providing service?
• Percentage of satisfied users measures user satisfaction: Can you provide better FAQs to improve user satisfaction scores in the next quarter?

It’s important to make sure that your KPI’s are aligned with each other to avoid unintended consequences later. For instance, you may announce happily that your ticket closures are more efficient, so you are saving money on the service. Looking at the next quarter’s KPI’s, you then discover that you’ve angered your users by closing tickets without solving the real problem and therefore your satisfaction scores have plummeted.

7 Steps to Get Started with Key Performance Indicators

1. Ensure your team understands KPI’s. You’ll be an expert once you read the information linked below in “Now It’s Your Turn.”
2. Determine what indicators are important. What are those vital few indicators that tell you things are working as intended?
3. Assess which data elements can be collected easily. If it’s too difficult, you will lose interest and stop.
4. Express the KPI as a formula (e.g. cost/searches=cost per search) and make sure you calculate it the same way every time.
5. Create a simple “dashboard” — a place to track the data on your internal website or on a shared Excel document.
6. Discuss the results every month with your team and make changes to your KPI’s until you are satisfied they are correct.
7. Dig down to analyze the KPI’s for your planning efforts.

KPI’s are one measurement technique in your arsenal. They can be a quick and useful tool to let you diagnose strengths and weaknesses in your process, make strategic decisions, and ensure you are heading in the right direction. Don’t forget: the real value is in the discussion of results with your team, not the numbers themselves.

“Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted.” –
Albert Einstein

Now It’s Your Turn

1. Read these articles to learn more about KPIs:

• The basics of KPIs: Key Performance Indicators for Nonprofits: An Introduction (PDF)

• Advice on web KPI’s from Avinash Kaushik of the blog Occam’s Razor:

• Six Web Metrics/KPI’s to Die For

• Four Not Useful Web Analytics KPI Measurement Techniques

• Especially for Libraries: Library Key Performance Indicators from Radboud University Nijmegen (Netherlands)

2. Look at what other institutions are doing: check out the KPI Library website and sign up for their free newsletter.

3. Begin to develop KPI’s within your group or service.

Read more posts from Your Life@Work.

First up: our guidelines. When we decided that CDL’s central account would follow CDL individuals, there was some concern that we would end up with content like “I’m taking out the trash” or “I just saw the most amazing [fill-in-the-blank]” in CDL’s Twitter stream.

So, we agreed that a very small number of guidelines would be useful. We read the LowriderLibrarian post about using Twitter for organizations, and we drew inspiration, if not detail, from it. We used a wiki to write down our ideas, comment on each other’s changes, and tweak (and re-tweak). In the end, here is what we came up with:

Group account:  When you are tweeting from a group account (e.g. eScholarship, Calisphere, Discovery & Delivery, CDL_WAS, etc.), remember that you are representing CDL and the University of California.
Individual account:  If you choose to be followed by the CDL as an individual tweeter, remember that your comments will reflect on CDL. Think of this as your professional persona; the viewpoints and topics you express should be primarily professional in nature. If you want to use Twitter primarily to express your personal views and discuss personal topics, maintain a separate personal account for this purpose. (Optional:  you may identify yourself as a CDL employee in your Twitter profile, i.e. in the bio or the username.) Note:  CDL reserves the right to stop following any individual account at any time.
In general:  remember that your tweets may be forwarded and shared by others in the Twitter community–in fact, we hope they will be!

At this point, only 6 individual CDL employees have chosen to be followed by the central CDL account, so it may be the case that the personal/professional guideline didn’t quite hit the mark. Stay tuned.

As for assessing the performance of CDL’s Twitter presence, I’ll organize this information in terms of how it addresses our original reasons for setting up the Twitter account.

Our first goal was to gain a new way to get our newsletter, CDLInfo, out into the world. To test whether or not this is succeeding, I’ve done a little comparison between the email list of subscribers to CDLInfo with the Twitter account’s followers and asked, ”Is Twitter helping us to reach new people?”  The results of the comparison are that only 5% of the Twitter followers appear to be duplicates of the email list members, with another 8% having Twitter names that make them unidentifiable, so that the ratio could conceivably rise to as high as 13%.

Our second goal was to amplify the voices and messages of the accounts it follows. This is more true for some accounts than others. For the Twitter accounts with fewer followers than CDL–178 at the time of this writing–CDL’s reach absolutely makes a difference in getting the word out. I have also been tracking CDL’s performance using a tool called Twitalyzer over time, although Twitalyzer is normed against some very large “power users.” Twitalyzer has defined two metrics that have some relevance for this amplification notion: “Influence” and “Clout.” Influence, as they define it, is ”measured by the number of times you are retweeted.” Retweeting takes place when another Twitter user forwards one of your tweets. Clout is “simply the number of references to you divided by the total number of possible references (as governed by the Twitter Search APIs).” To date, CDL’s measures against both of these metrics are very low.

Our third goal was to increase CDL’s visibility, in concert with our values of openness and sharing. In order to assess the account’s performance against this goal, we have two sources of information. On the one hand, we can look at the WordPress website statistics and ask whether or not Twitter, in general, and the CDL Twitter account specifically, is sending traffic to the CDLINFO page. In addition, we can ask whether or not the CDL Twitter account is gaining in popularity, thereby providing a net increase in the number of readers for CDLINFO.

For the past 30 days, Twitter.com is the top referrer to the CDLINFO page. Twitter.com/caldiglib is the 4th highest referrer for the same period. And, since its inception, in October, the CDL Twitter account has gained followers steadily, as the chart above shows. The distinction between Twitter.com and Twitter.com/caldiglib as referrers can be explained in this way: this is the difference between someone following a link from his or her own Twitter home page, which appears as http://twitter.com, or that same person visiting CDL’s Twitter home page, which appears as http://twitter.com/caldiglib.

To sum up, then, CDL’s Twitter account is getting the CDLInfo newsletter to new people and increasing CDL’s visibility generally.

UC’s May 09 Annual Accountability Report is now available online at http://www.universityofcalifornia.edu/accountability/.  The website provides a spotlight on accountability issues from President Yudof, an introduction to the report, and a discussion of key themes and trends.  From the website you can print out the entire report, access individual indicators or chapters, get background information and technical details about each indicator, and access the underlying data.  You can also print out and read individual campus portraits and campus accountability profiles

Libraries Chapter

The Libraries chapter, http://www.universityofcalifornia.edu/accountability/index/chapter/10, contains five indicators that measure UC library “size, impact and vitality.”  Three of the five indicators provide the same comparison institutions that are used in the rest of the Accountability Report.  The indicators are:

• Association of Research Libraries Rankings of Campus Libraries – UC and Comparison Institutions, 2005 to 2007
• Library Holdings in Volumes by Campus – UC and Comparison Institutions, 2007
• Use of Interlibrary Loan (ILL) within the UC System, 2000-01 to 2007-08
• Electronic Journal Usage within the UC System, 2005-06 to 2007-08
• Students Participating in Library Classes, Tours and Presentations – UC and Comparison Institutions, 2004-05 to 2006-07

For the 2009 report, CDL staff prepared the report with University Librarian overview in a severely constrained timeframe, necessitating a selection of indicators based on existing data.  The result is a strong, but somewhat limited picture of the full work of the libraries.

Supplementary Report and Next Steps

Because the Libraries chapter could provide only this limited view, the University Librarians have exercised the option to submit a Supplementary Report that will allow fuller control over length and timing.  The team assembled for this effort can address the question of what story we want to tell the public about the libraries and present the data accordingly.

The team members are: Ruth Jackson, University Librarian, UC Riverside; Susan Parker, Deputy University Librarian, UC Los Angeles; Dolores Davies, Director of Communications, UC San Diego; and Joanne Miller, Library Planning Analyst, California Digital Library.

For more information, please contact Joanne Miller (joanne.miller@ucop.edu).