I am guest-editing the Special Issue on Utility Based Data Mining with Gary Weiss and Bianca Zadrozny in Data Mining and Knowledge Discovery.  Call For Papers

I Co-chaired The First and Second ACM SIGKDD Workshops on Utility-Based Data Mining in 2005 & 2006 with Gary Weiss and Bianca Zadrozy:

• The Second ACM SIGKDD Workshop on Utility-Based Data Mining, Chicago, Illinois , 2005

  • Workshop website

  •  UBDM-05: Workshop on Utility-Based Data Mining. SIGKDD Explorations 7(2): 145-147, 2005.

• The Second ACM SIGKDD Workshop on Utility-Based Data Mining, August 2006, Philadelphia, Pennsylvania.

  • Workshop Website

  •  UBDM 2006: Utility-Based Data Mining 2006 Workshop Report. SIGKDD Explorations, 8(2), 2006.

NSF Grant: STTR Program (with Daniele Micci-Barreca): "Active Learning System for Audit Selection"

• Saar-Tsechansky Maytal and Provost Foster. “Handling Missing Values When Applying Classification Models”. Journal of Machine Learning Research, 8(Jul):1623--1657, 2007.

Abstract:

Much work has studied the effect of different treatments of missing values on model induction, but little work has analyzed treatments for the common case of missing values at prediction time. This paper first compares several different methods---predictive value imputation, the distribution-based imputation used by C4.5, and using reduced models---for applying classification trees to instances with missing values (and also shows evidence that the results generalize to bagged trees and to logistic regression). The results show that for the two most popular treatments, each is preferable under different conditions. Strikingly the reduced-models approach, seldom mentioned or used, consistently outperforms the other two methods, sometimes by a large margin. The lack of attention to reduced modeling may be due in part to its (perceived) expense in terms of computation or storage. Therefore, we then introduce and evaluate alternative, hybrid approaches that allow users to balance between more accurate but computationally expensive reduced modeling and the other, less accurate but less computationally expensive treatments. The results show that the hybrid methods can scale gracefully to the amount of investment in computation/storage, and that they outperform imputation even for small investments

• Paul Tetlock, Maytal Saar-Tsechansky and Sofus Macskassy. “More Than Words: Quantifying Language to Measure Firms' Fundamentals”.  Journal of Finance, Forthcoming.

Abstract:

We examine whether a simple quantitative measure of language can be used to predict individual firms’ accounting earnings and stock returns. Our three main findings are: (1) the fraction of negative words in firm-specific news stories forecasts low firm earnings; (2) firms’ stock prices briefly underreact to the information embedded in negative words; and (3) the earnings and return predictability from negative words is largest for the stories that focus on fundamentals. Together these findings suggest that linguistic media content captures otherwise hard-to-quantify aspects of firms’ fundamentals, which investors quickly incorporate into stock prices.

• Saar-Tsechansky Maytal and Provost Foster. “Decision-centric Active Learning of Binary-Outcome Models”, Information Systems Research, Vol. 18, No. 1, pp. 1–19, 2007.

Abstract:

It can be expensive to acquire the data required for businesses to employ data-driven predictive modeling, for example to model consumer preferences to optimize targeting. Prior research has introduced “active learning” policies for identifying data that are particularly useful for model induction, with the goal of decreasing the statistical error for a given acquisition cost (error-centric approaches).  However, predictive models are used as part of a decision-making process, and costly improvements in model accuracy do not always result in better decisions. This paper introduces a new approach for active data acquisition that targets decision-making specifically.  The new decision-centric approach departs from traditional active learning by placing emphasis on acquisitions that are more likely to affect decision-making. We describe two different types of decision-centric techniques.  Next, using direct-marketing data, we compare various data-acquisition techniques.  We demonstrate that strategies for reducing statistical error can be wasteful in a decision-making context, and show that one decision-centric technique in particular can improve targeting decisions significantly.  We also show that this method is robust in the face of decreasing quality of utility estimations, eventually converging to uniform random sampling, and that it can be extended to situations where different data acquisitions have different costs.  The results suggest that businesses should consider modifying their strategies for acquiring information through normal business transactions. For example, a firm such as Amazon.com that models consumer preferences for customized marketing may accelerate learning by proactively offering recommendations—not merely to induce immediate sales, but for improving recommendations in the future.

• Saar-Tsechansky Maytal and Provost Foster. “Active Sampling for Class Probability Estimation and Ranking.” Machine Learning, 54:2, 153-178, 2004

Abstract:

In many cost-sensitive environments class probability estimates are used by decision makers to evaluate the expected utility from a set of alternatives. Supervised learning can be used to build class probability estimates; however, it often is very costly to obtain training data with class labels. Active learning acquires data incrementally, at each phase identifying especially useful additional data for labeling, and can be used to economize on examples needed for learning. We outline the critical features of an active learner and present a sampling-based active learning method for estimating class probabilities and class-based rankings. BOOTSTRAP-LV identifies particularly informative new data for learning based on the variance in probability estimates, and uses weighted sampling to account for a potential example’s informative value for the rest of the input space. We show empirically that the method reduces the number of data items that must be obtained and labeled, across a wide variety of domains. We investigate the contribution of the components of the algorithm and show that each provides valuable information to help identify informative examples. We also compare BOOTSTRAP-LV with UNCERTAINTY SAMPLING, an existing active learning method designed to maximize classification accuracy. The results show that BOOTSTRAP-LV uses fewer examples to exhibit a certain estimation accuracy and provide insights to the behavior of the algorithms. Finally, we experiment with another new active sampling algorithm drawing from both UNCERTAINTY SAMPLING and BOOTSTRAP-LV and show that it is significantly more competitive with BOOTSTRAP-LV compared to UNCERTAINTY SAMPLING. The analysis suggests more general implications for improving existing active sampling algorithms for classification.

• Saar-Tsechansky Maytal, Pliskin Nava, Rabinowitz Gadi., and Porath Avi, "Mining Relational Patterns from Multiple Relational Tables," Decision Support Systems, Vol. 27, No. 1-2, 177-195, 1999.  An earlier version appeared in HICSS 2001.