Model Free Reinforcement Learning Based Controller For Grid-tied 9-Level Packed-E-Cell Multi-level Inverter

This paper proposes a model-free reinforcement learning-based controller (RLC) for single phase grid connected 9-level packed-E-Cell (PEC9) multilevel inverter (MLI). The RLC design consists of actor-critic architecture which combines the value-based and policy-based learning methods using the stable learning algorithm, proximal policy optimization (PPO). In the system under study, the control objectives are regulating the two capacitors' voltages around their reference values and generating a sinusoidal current with reduced total harmonic distortion (THD). The training environment is designed in MATLAB/Simulink in which different variations in the voltage and current references are included. The testing results showed the capability of the designed RLC to generate 5A and 10A current signals with 1.9% and 1.3%, respectively, while the two capacitors voltage error were kept below 1.5 V. The dynamic response is also investigated in the case of having a variation in the reference phase shift or the DC voltage source. The robustness of the proposed control is tested in the case of voltage grid sag and swell scenarios. Furthermore, the performance of the RLC is compared with the results of finite control set - model predictive control (FCS-MPC) to show the RLC competitiveness in fulfilling the control objectives simultaneously.