STATISTICAL AUDITING OF NON-TRANSPARENT EXPERT ASSESSMENTS

  • 31/03/2022
Đăng nhập để xem được nhiều hơn
Giới thiệu

A STATISTICAL MODEL IS DEVELOPED FOR AN AUDIT OF THE ASSESSMENTS OF A PANEL OF EXPERTS WHEN LITTLE INFORMATION IS MADE AVAILABLE BEYOND A FINAL ANNOUNCEMENT OF THE INDIVIDUAL ASSESSED RATINGS GIVEN. THE APPLICATION IS TO THE PROCESS FOR THE RESEARCH ASSESSMENT EXERCISE FOR UK UNIVERSITIES. BASED ON THE PROPORTIONS OF THE PUBLICATIONS A PANEL DEEMS TO BE INTERNATIONAL STANDARD, NATIONAL STANDARD OR UNCLASSIFIED, A DEPARTMENTS RESEARCH OUTPUT IS RATED BY THE PANEL ON A SEVEN POINT SCALE. THE EXPERT PANELS REMIT IS CAREFULLY INTERPRETED AND THE GIVEN RATINGS ARE MODELLED VIA AN UNDERLYING TRINOMIAL RANDOM VARIABLE WITH A BIVARIATE NORMAL APPROXIMATION. A LIKELIHOOD FUNCTION IS DEVELOPED AND MAXIMISED IN ORDER TO OBTAIN FITTED RATINGS FOR ALL UNITS OF ASSESSMENT. THE MODELS FITTED VALUES FOR THE GIVEN RATINGS EXPLAIN OUTCOMES REMARKABLY WELL AND THERE ARE FEW MIS-CLASSIFICATIONS; BUT THERE ARE SOME SURPRISING OUTLIERS THAT DO STILL REQUIRE SOME EXPLANATION. THE PROCEDURE ILLUSTRATES WELL HOW STATISTICIANS, SURPRISINGLY, MIGHT BE ABLE TO MODEL AND AUDIT FOR CONSISTENCY THE WORK OF EXPERTS EVEN IF LITTLE OR NO INFORMATION IS PROVIDED, BEYOND VAGUE PRIOR PUBLISHED GUIDELINES FOR THE ASSESSMENTS AND THE FINAL RATINGS GIVEN. 2016, INDIAN STATISTICAL INSTITUTE.

Trả lời

Email của bạn sẽ không được hiển thị công khai. Các trường bắt buộc được đánh dấu *

TÀI LIỆU KHÁC