Lightweight discrete q-learning for self-tuning PID on ESP32: Robustness evaluation and cross-volume adaptation in egg incubators

Abstract

Temperature stability is the most crucial factor in the success of the egg incubation process. The use of conventional Proportional–Integral–Derivative (PID) control with static Ziegler–Nichols tuning often fails to adapt to external disturbances and thermal dynamics, leading to temperature overshoot that can be fatal to embryo survival. This study proposes the implementation of an adaptive PID controller using a Discrete Q-Learning method based on Edge-AI on an ESP32 microcontroller. Experimental results under standard conditions show that the Q-Learning method successfully reduces overshoot by up to 81.8%, limiting the temperature spike to only 0.2°C above the target of 38.0°C, and accelerating the stabilization time by 76.9% with a reduction in IAE of 52.5%. In the dynamic disturbance rejection test, the adaptive system validated the algorithm's robustness against dynamic disturbances. Furthermore, cross-environment adaptation evaluation by reducing the incubator volume by 50% demonstrates the agent’s autonomous adaptation capability, eliminating overshoot entirely (0.000°C) without parameter recalibration and reducing IAE by 55.1% compared to static PID. This study concludes that the implementation of Q-Learning on low-cost hardware produces a robust, precise, and autonomously adaptive thermal control system for agricultural technology applications.