Design and implementation of an authenticated post-quantum session protocol using ML-KEM (Kyber), ML-DSA (Dilithium), and AES-256-GCM

Abstract

Introduction: Session establishment, the process by which two parties authenticate each other and derive a shared secret key, forms the foundation for secure digital communication. Quantum computers threaten this foundation by breaking classical public-key primitives such as RSA and elliptic-curve Diffie–Hellman (ECDH), thereby enabling harvest-now–decrypt-later (HNDL) attacks that endanger long-term confidentiality.
Methods: This paper presents the design, implementation, and evaluation of an authenticated, quantum-resistant session protocol that replaces these vulnerable mechanisms with their post-quantum counterparts. The proposed protocol integrates ML- KEM-1024 (FIPS 203; CRYSTALS, Kyber) for ephemeral key exchange, ML-DSA-65 (FIPS 204; CRYSTALS, Dilithium) for endpoint authentication, and AES-256-GCM for symmetric protection. A transcriptbound HKDF–SHA3-256 key schedule and a 96-bit GCM nonce construction with conservative rekey limits are used to ensure forward secrecy, downgrade resistance, and message integrity. A Python/C prototype (PQClean MLKEM-1024 with PyCryptodome AES-256-GCM) was benchmarked over 1,000 iterations on commodity hardware.
Results: The results show that sub-millisecond cryptographic overhead MLKEM-1024 matches the performance of X25519 while vastly outperforming RSA-3072 in secure session establishment, and symmetric encryption remains cost effective. Nonces are unique 96-bit values, never reused across directions or beyond 2³² records, following NIST SP 800-38D; when nonce-misuse resistance is required, AES-256-GCM-SIV (RFC 8452) is supported as a drop-in alternative. Empirical tests under both local and WAN-emulated (≈ 40 ms RTT) network conditions confirm that the additional post-quantum cost maintains the handshake cryptographic latency in the 0.50–0.70 ms range.
Discussion: These results demonstrate that fully authenticated, forward-secure, quantum-resistant session negotiation is practical for real-world deployments.