Our research group at the University of Michigan has been working on the "related modules" block for Drupal.org for more than 2 years now. We have published 2 papers on this project so far:
1) Assessment of Conversation Co-mentions as a Resource for Software Module Recommendation. Will be presented at ACM Recommender System Conference'09
2) Conversation Pivots and Double Pivots. Presented at ACM Computer Human Interaction Conference'08
This paper is submitted to ACM Recommender System Conference'09, co-authored with my advisor Prof. Paul Resnick.
ABSTRACT
A brief history to begin with ...
The idea is to generate "related modules" recommendation based on co-citations. Suppose we have TinyMCE and FCKeditor co-mentioned together in many forum discussions, then we consider the 2 modules related to some extend. Here is a detailed explanation.
Previously, 'related modules' were generated based on discussions in d.o. forum -- if several modules were mentioned in the same discussion threads, we consider them to be somewhat related. (More detailed explanation of the algorithms can be found in my previous Planet Drupal blogs).
We developed 4 module recommendation algorithms and tested them on Drupal.org. And we used Google Analytics and tracked the click-through rates. The overall click-through rate was 0.263%, co-occurrences 0.097%, relevance 0.141%, recency 0.114% and uniqueness 0.138%. The relevancy algorithm appeared to have the highest click-through rate, but it was only significantly higher than the co-occurrences algorithm.
You might have noticed that the pivots_block is enabled on drupal.org module pages, such as http://drupal.org/project/i18n. This blog is trying to explain the algorithms we are using in the pivots recommendation system.
We hope to deploy the pivots block to Drupal.org recently. The block displays on a module page its related discussions and related modules. This article explains how the pivots block will be deployed on Druapl.org.
At the heart of the pivots Drupal module recommendation system are the secret recommender algorithms. Currently we are playing with 4 algorithms.
We now have two more weeks of usage data on authenticated D.O. users from 7/28/2008 to 8/11/2008. The data shows a steady pattern of usage overtime. Please refer to the attachment for a brief report. We are now quite confident that the pivots block has certain values to users in terms of module recommendation.
Previously we computed the "related projects" by using the frequency of project co-mentions in discussions. For example, if the module Fivestar and CCK were mentioned together in 33 different discussion threads, whereas Fivestar and jRating in 10 threads, then we would think CCK was more relevant to Fivestar than jRating because it has more co-mentions.
However, that algorithm ignores the fact that some very popular modules like CCK simply have more chances to get mentioned, even if it's not that relevant in the context.
