Trustworthiness Calibration Framework for Phishing Email Detection Using Large Language Models
Published 6 Nov 2025 · arXiv · Daniyal Ganiuly
Overview
The paper presents the Trustworthiness Calibration Framework (TCF) for evaluating phishing email detection systems using large language models (LLMs). It emphasizes the need for trust-aware evaluation beyond mere accuracy.
Key Insights
- Trustworthiness Calibration Framework (TCF): A methodology to assess phishing detectors across calibration, consistency, and robustness.
- Trustworthiness Calibration Index (TCI): A bounded index measuring trustworthiness.
- Cross-Dataset Stability (CDS): A metric quantifying stability across datasets.
- Performance: GPT-4 outperforms LLaMA-3-8B and DeBERTa-v3-base in trust profile.
- Statistical Analysis: Reliability varies independently of raw accuracy.
BFSI Relevance
- Why Relevant: Phishing detection is critical for cybersecurity in BFSI, impacting data protection and fraud prevention.
- Primary Sector: Financial Services
- Subsectors: Cybersecurity
- Actionable Implications: BFSI professionals should integrate trust-aware evaluation frameworks like TCF to enhance phishing detection systems.
researcher peer-reviewed-paper global