Tuesday, May 3, 2011

Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability

Title: Better Hypothesis Testing for Statistical Machine Translation:
Controlling for Optimizer Instability
Speaker: Jonathan Clark
When: Tuesday, 4/19 at Noon

Abstract:
In statistical machine translation, a researcher seeks to determine
whether some innovation (e.g., a new feature, model, or inference
algorithm) improves translation quality in comparison to a baseline
system. To answer this question, he runs an experiment to evaluate the
behavior of the two systems on held-out data. In this paper, we
consider how to make such experiments more statistically reliable. We
provide a systematic analysis of the effects of optimizer instability
(an extraneous variable that is seldom controlled for) on experimental
outcomes, and make recommendations for reporting results more
accurately.

This is joint work with Chris Dyer, Alon Lavie, and Noah Smith. It was
recently accepted for publication as an ACL short paper.

2 comments:

sameena Khan said...

for correct translation of english, chinese, japanese, italian, urdu, etc just visit to
Degree translation

Quincy said...

Awesome!