TY - JOUR

T1 - Dealing with collinearity in finite impulse response models using Bayesian shrinkage

AU - Nounou, Mohamed N.

PY - 2006/1/4

Y1 - 2006/1/4

N2 - Finite impulse response (FIR) models are widely used in the chemical industry due to their simplicity and easily defined model structures. However, they usually require a large number of coefficients to capture the process dynamics. This large number of coefficients introduces collinearity or redundancy in the models, which usually increases the uncertainty of their estimation. This paper presents a Bayesian approach, called empirical Bayesian FIR (EBFIR) modeling, to deal with this collinearity problem. Unlike ridge regression (RR), which shrinks the FIR model parameters toward zero in order to reduce their variations, EBFIR makes better use of the data by estimating a prior density function for the FIR coefficients, which is then used within a Bayesian framework to shrink the FIR coefficients in a more rigorous fashion. The developed Bayesian approach is computationally inexpensive, as a closed form solution of the Bayesian estimate of the FIR coefficients is derived. The EBFIR approach is also more general than some of the existing methods, such as ordinary least squares regression (OLS) and RR, which are shown to be special cases of EBFIR. Finally, the developed EBFIR approach has been shown to outperform some of the commonly used existing FIR estimation methods over a wide range of noise levels.

AB - Finite impulse response (FIR) models are widely used in the chemical industry due to their simplicity and easily defined model structures. However, they usually require a large number of coefficients to capture the process dynamics. This large number of coefficients introduces collinearity or redundancy in the models, which usually increases the uncertainty of their estimation. This paper presents a Bayesian approach, called empirical Bayesian FIR (EBFIR) modeling, to deal with this collinearity problem. Unlike ridge regression (RR), which shrinks the FIR model parameters toward zero in order to reduce their variations, EBFIR makes better use of the data by estimating a prior density function for the FIR coefficients, which is then used within a Bayesian framework to shrink the FIR coefficients in a more rigorous fashion. The developed Bayesian approach is computationally inexpensive, as a closed form solution of the Bayesian estimate of the FIR coefficients is derived. The EBFIR approach is also more general than some of the existing methods, such as ordinary least squares regression (OLS) and RR, which are shown to be special cases of EBFIR. Finally, the developed EBFIR approach has been shown to outperform some of the commonly used existing FIR estimation methods over a wide range of noise levels.

UR - http://www.scopus.com/inward/record.url?scp=30444435938&partnerID=8YFLogxK

U2 - 10.1021/ie048897m

DO - 10.1021/ie048897m

M3 - Article

AN - SCOPUS:30444435938

SN - 0888-5885

VL - 45

SP - 292

EP - 298

JO - Industrial and Engineering Chemistry Research

JF - Industrial and Engineering Chemistry Research

IS - 1

ER -