Can You Trust Your LLM Judge? A Calibration Lab

Test an LLM judge for order, verbosity, format, citation, and correctness bias using blinded human labels, swap tests, confusion matrices, and bootstrap intervals.