I tweeted this question earlier: are there studies that show ways to measure social learning? More specifically, are there studies or research that shows ways to measure the ROI of using social media tools and processes for corporate education?

I know there are posts on ideas of how social media should be used for education. For instance, Tony Karrer recently had a post on the questions that should be asked when doing an analysis for implementing eLearning 2.0:

Content

• What content is already shared through other means? Ex. are lessons learned discussed, or work-arounds.
• Is there information that can be created and shared coming from either a 3rd party (e.g., a help desk, experts, etc.) or from the audience itself?
• What content gets updated more frequently?
• What reference material is already being created that might be a target?

Audiences

• Who has the pain?
• Who’s going through an experience that they would want to share?
• Who is able and active enough to use the tools to create content?
• Does it align with their motivation or can it be aligned with their motivation?
• Are there natural content creators that we could leverage?

Kevin Jones at Engaged learning listed 15 objections to using social media as a learning tool. Objection 13 is “How do you measure ROI?”. Jay Cross gives the business case for informal learning, saying that business that don’t design for informal learning leave money on the table (long post, but lots of good info).

Here are some specific examples I found:

• Tony Karrer posted a link to an interview with GE’s CIO, who said their internal social media skills locator site gets 25 million hits a day. Is that a good measure of ROI for learning – hits per day?
• An article in the Harvard Business Review reports on research that proves the competitive imperative of learning. The article points out that “great execution is difficult to sustain—not because people get tired of working hard, but because the managerial mind-set that enables efficient execution inhibits employees’ ability to learn and innovate. A focus on getting things done, and done right, crowds out the experimentation and reflection vital to sustainable success.”
  The article talks about Learning as Execution. There may be examples later in the article, I can’t afford it though (you have to pay to read more than the 1st page).
• The Cold Fusion Developer’s Network site has an article about how a design firm switched from email to opensource, web 2.0 SaaS (software as a service) delivered solutions, with 2K a month in productivity gains.
• An article in Training Magazine entitled University 2.0 (Sep2007, Vol. 44 Issue 8, p22-24, 3p) suggests the following ways to measure:
  “Measure how the money you spent led to efficiencies, better performance, or valuable talent development (e.g how much you were able to accelerate the development of new consultants so they’re able to bill their hours sooner)”
  ROI may not be the best evaluation. “Instead, measurements around actionable performance indicators make more sense”
• An article in Training and Development entitled The World According to Wiki (2007 vol.61 iss.5 pg.28) has stories of how organizations are using wikis, but really don’t give ways to measure success.

So, how do you measure success when using social media tools for learning? Or, instead of measuring the tools, do we measure the success of the communities using the tools? Or is should we measure a combination of the two – the success of the communities using the tool along with the effectiveness of the tools? What does everyone think?

And does anyone have any links to studies on measurement?