adaptive image

On Imbalance in Case Types: Evaluating and Enhancing PLMs for Criminal Court View Generation

1Hunan University, China 2Nanyang Technological University, Singapore

TNNLS 2025

Abstract

Criminal Court View Generation (CCVG) task aims to produce succinct and coherent summaries of fact descriptions, providing interpretable opinions for verdicts. Traditional text generation evaluation metrics, such as ROUGE, BLEU, and BERTSCORE, are extensively employed for this task and measure performance by averaging the assessment scores of all samples within the test set. However, these sample-averaged metrics encounter two primarily dilemmas: 1) they fail to fairly assess overall evaluation scores across different case types, and 2) they overlook the measurement of the degree of performance imbalance between case types. To fill this research gap, we propose two novel case-type-oriented evaluation metrics: Case-type-oriented Text Generation (CTG) and Case-type-oriented Imbalance Performance (CIP). First, CTG mitigates the unfair assessment among different case types by assigning equal weight to each type. Second, CIP evaluates performance imbalance by measuring the distance between the performance of each case type and the overall performance. We provide three Theorems to elucidate the properties of CIP, demonstrating that CIP can effectively identify the extent to which a CCVG model achieves balanced generation performance across different case types. Furthermore, we propose an embarrassingly simple and effective Charge-Guided Encoder-Decoder (CGED) framework to enhance performance fairly across different case types in encoder-decoder pre-trained language models. Code is available at: https://yuquanle.github.io/Case-type-oriented-metrics-homepage/.

Case-type-oriented Text Generation evaluation metrics

Experiment

BibTeX

  @article{le2025cto,
  title={On Imbalance in Case Types: Evaluating and Enhancing PLMs for Criminal Court View Generation},
  author={Le, Yuquan and Xiao, Zheng and Ding, Yan and Chng, Eng Siong and Li, Kenli},
  journal={TNNLS},
  year={2025}
}