Thursday, 30 January 2014

Hidden Rule Changes



The ARC has recently released consultation documents for the 2015 ERA submission. The process is largely unchanged from the 2012 rules. However sharp eyed readers will have noticed business rules 131 and 132. These rules place a limit on the percentage of publication outputs that can be recoded from the journal discipline codes assigned by the ARC. The recoding percentage is capped at 40%. This presumably has been introduced to limit gaming by institutions trying to hide supposedly poor quality or low citing publications. It could potentially have major impacts for smaller institutions doing multidisciplinary research who may struggle to research assessment thresholds. Its hard to believe this important change was accidentally omitted from the change summary in the submission guidelines – perhaps the rule changes were considered and discarded but accidentally left in the business rules document.

I suspect most Australian universities did their very best to present the best possible submissions in 2010 and 2012 but its not clear that gaming significantly changed outcomes. Its interesting to look at the 2012 outcomes in the 15xx codes. A number of universities submitted assessable numbers of publications in 1599, the 'not else where classified' subdivision of the Commerce, Management, Tourism and Services (CMTS) group. Of the 15xx codes submitted by these institutions, who were presumably hiding publications in 1599, only one of the defined 15xx codes scored a 4 ('above world standard'), and handful scored 3 ('world standard'). The only success from this strategy was probably to divert and consolidate some poor outcomes into a single code rather than advertising apparent shortcomings in the economically important CMTS disciplines.

No comments:

Post a Comment