关于TOEFL作文电脑评分(Automated Scoring)的改革

标签:
toefl作文改革机器评分ibt教育 |
分类: 学海无涯 |
早上查收邮件收到一封ETS发来一个问卷,主要内容是关于对TOEFL IBT作文采用电脑评分(Automated Scoring)的一些内容。
看到第一部分我还是感到比较震惊,原来independent essay部分他们已经开始采用电脑评分了,之前都没有意识到。上网一查才知道从2010年开始就开始用机器评分e-rater了。
接下来的问卷主要是关于对IBT作文人工评分和机器评分的优缺点,以及人工评分和机器评分各自需对应对写作的哪些方面侧重考查,以及被调查者对电脑评分改革的看法之类。
关于对机器评分改革的看法,我基本持反对态度。因为感觉语言这东西是活的,而机器怎么说也是死的程序。我认为要做一个较好的机器评分系统并非易事,在评估时将会涉及到大量的字符串方面的相关算法,比如正则表达式匹配之类,而且评分时考查的方面很多,字符串方面的算法集将会十分复杂和庞大;另外,要对一篇essay进行评估,必然需要有一个比对标准,当然对单词拼写、包括语法之类的简单的方面的检查技术已比较成熟,后台找个较好的词库之类进行匹配检查应该可以做到,但是鉴于语言表达过于灵活,最后可能还是会出些问题,比如我们常用的word的语法检查功能有时就检查不出来一些语法错误,最重要的是文章的一些修辞以及文章的整体结构之类比较抽象的东西仅仅通过电脑很难进行考察,人工智能ms还没到那个程度吧,但这也并非绝对,倘若ETS与一些米国人工智能方面突出的牛校合作开发智能评分系统,经过不断的学习,可能也会做个不错的评分系统。我不清楚现在人工智能在这方面的情况,因而也不敢妄断。总之我是反对其采用完全的电脑评分。
问卷中一些人工评分和机器评分的原则我感觉对大家准备IBT作文还是一个很好的参考,在这里我特意截了下图,分享给大家,希望对大家备考有些帮助。
For the TOEFL writing section, you were required to write two essays. Were you aware that the independent essay – the one that requires you to state, explain, and support your opinion about an issue – would be scored by both a human reader and a computer?
If you had known that a computer would score your essay, would you have changed how you prepared for the test or how you wrote your essay?
There are a number of ways that you
might have changed how you wrote your TOEFL essay because it would
be scored partly by computer
and partly by a human reader. Which of the following ways did you
(or might you) change how you wrote?
http://s14/middle/620583b448dbe507ef89d&690Scoring)的改革" TITLE="关于TOEFL作文电脑评分(Automated
Because both a computer and a human reader would score your essay, are there any other ways that you might have changed how you wrote your TOEFL essay?
Following are some reasons that have been given for using automated (computer) scoring of test takers’ essays. Please indicate how much you agree or disagree that each reason is a GOOD one for using automated scoring.
http://s6/middle/620583b448dbe5b5bff95&690Scoring)的改革" TITLE="关于TOEFL作文电脑评分(Automated
Following are some reasons that have been given for NOT using automated (computer) scoring test takers’ essays. Please indicate how much you agree or disagree that each reason is a GOOD one for NOT using automated scoring.
http://s14/middle/620583b448dbe5e6f8e1d&690Scoring)的改革" TITLE="关于TOEFL作文电脑评分(Automated
Following are some factors that are
considered in the evaluation of TOEFL essays. How important do you
think each factor is for
computer scoring?
http://s12/middle/620583b448dbe6118d16b&690Scoring)的改革" TITLE="关于TOEFL作文电脑评分(Automated
Following are some factors that are considered in the evaluation of TOEFL essays. How important do you think each factor is in scoring by human readers?
http://s2/middle/620583b40749309f18b81&690Scoring)的改革" TITLE="关于TOEFL作文电脑评分(Automated
Please indicate how much you agree or
disagree with each of the following statements.A good way to
evaluate TOEFL essays is to:
http://s3/middle/620583b448dbe64e29742&690Scoring)的改革" TITLE="关于TOEFL作文电脑评分(Automated