Systems

An input-output system is defined by an input signal set $\mathcal{U}$, an output set $\mathcal{Y}$ and a subset $\mathcal{R} \subset \mathcal{U} \times \mathcal{Y}$, called the rule (relation) of the system. Hence, $\mathcal{R}$ consists of the input-output pairs in the system. Associated with such an input-output relation $\mathcal{R}$ is a transformation or map $\mathcal{T}$ such that $y = \mathcal{T}(u)$, and thus

$\mathcal{T} : \mathcal{U} \ni u \mapsto \mathcal{T}(u) = y \in \mathcal{Y},$

and thus

$\mathcal{R} = \{(u, \mathcal{T}(u)),\quad u \in \mathcal{U}\}.$

Let $T_1, T_2$ be time-index sets; and $U, Y$ be signal range spaces such that $\mathcal{U} = U^{T_1}, \mathcal{Y} = Y^{T_2}$, that is:

$\mathcal{U} = \{f : T_1 \to U\},$

$\mathcal{Y} = \{f : T_2 \to Y\}.$

If $\mathcal{U}$ and $\mathcal{Y}$ consist of signals with discrete-time indices, then the system is said to be a discrete-time (DT) system. If the indices are both continuous, then the system is a continuous-time (CT) system. If one of them is discrete and the other continuous, the system is said to be hybrid. Often, we have $T = T_1 = T_2$, which will be assumed in the following.

---

System Properties

---

Linear Systems

Linear systems have important engineering practice. Many physical systems are locally linear, as we have seen earlier.

An input-output system is linear if $\mathcal{U}$, $\mathcal{Y}$, $\mathcal{R}$ are all linear vector spaces. However, in the context of our course, we will have a more restrictive definition for linearity.

We note here that a precise characterization for linearity (for a system as in ) would require the interpretation of a system as a (bounded) linear operator from one space to another space. One can obtain a Riesz representation theorem type characterization leading to , provided that $\mathcal{U}$ and $\mathcal{Y}$ satisfy certain properties, and the system is continuous and linear. The following discussion makes this explicit.

Let $\mathcal{T}$ be a linear system mapping $l_1(\mathbb{Z}; \mathbb{R})$ to $l_1(\mathbb{Z}; \mathbb{R})$. Let this system be linear and continuous; then the system can be written so that $y = \mathcal{T}(u)$ where:

$y(n) = \sum_{m \in \mathbb{Z}} h(n, m) u(m), \quad n \in \mathbb{Z}$

for some $h : \mathbb{Z} \times \mathbb{Z} \to \mathbb{R}$.

Building on this discussion, one takes the representation above as a definition of a linear system: In our course and in standard terminology in engineering and applied science, we generally say that a discrete-time (DT) system is linear if the input-output relation can be written as .

---

Linear and Time-Invariant (Convolution) Systems

If, in addition to linearity, we wish to have time-invariance, then one can show that

$y(n) = \sum_{k=-\infty}^{\infty} k(n, m) u(m),$

will have to be such that $k(n, m)$ should be dependent only on $n - m$. This follows from the fact that a shift in the input would have to lead to the same shift in the output, implying that $k(n, m) = k(n + \theta, m + \theta)$ for any $\theta \in \mathbb{Z}$.

Let us discuss this further. Suppose a linear system described by

$y(n) = \sum_m k(n, m) u(m), \qquad \text{}$

is time-invariant. Let, for some $\theta \in \mathbb{Z}$,

$v = \sigma^{-\theta}(u)$

so that $v(m) = u(m - \theta)$. Let the signal $g$ be the output of the system when the input is the discrete-time signal $v$. It follows that

$g(n) = \sum_{m \in \mathbb{Z}} k(n, m) v(m) = \sum_{m \in \mathbb{Z}} k(n, m) u(m - \theta) = \sum_{m' \in \mathbb{Z}} k(n, m' + \theta) u(m').$

By time-invariance, it must be that $g = \sigma^{-\theta}(y)$. That is, $g(n) = y(n - \theta)$ or $g(n + \theta) = y(n)$. Thus,

$g(n + \theta) = \sum_{m' \in \mathbb{Z}} k(n + \theta, m' + \theta) u(m') = y(n). \qquad \text{}$

Since the equivalence in - above has to hold for every input signal, it must be that $k(n + \theta, m + \theta) = k(n, m)$ for all $n, m$ values, and for all $\theta$ values. Therefore $k(n, m)$ should only be a function of the difference $n - m$. Hence, a linear system is time-invariant if and only if the input-output relation can be written as:

$y(n) = \sum_{m=-\infty}^{\infty} h(n - m) u(m)$

for some function $h : \mathbb{Z} \to \mathbb{R}$.

One can show that a convolution system is non-anticipative (causal) if $h(n) = 0$ for $n < 0$.

Similar discussions apply to continuous-time systems by replacing the summation with integrals:

$y(t) = \int_{\tau=-\infty}^{\infty} h(t - \tau) u(\tau)\,d\tau.$

Let $\delta_0$ be the generalized Dirac delta (impulse) function which we view as the limit of an approximate identity sequence (thus defining the Dirac delta distribution). Notably, if $h$ is continuous, it follows that

$\lim_{n \to \infty} \int h(t - \tau) \psi_n(\tau)\,d\tau = h(t).$

Thus, we have that when $u = \delta_0$,

$y(t) = \int_{\tau=-\infty}^{\infty} h(t - \tau)\delta_0(\tau)\,d\tau = h(t).$

Exercise

Let $x(t) \in \mathbb{R}^N$ and $t \geq 0$ and real-valued. Recall that the solution to the following differential equation:

$x'(t) = Ax(t) + Bu(t),$

$y(t) = Cx(t),$

with the initial condition $x(t_0) = x_0$ is given by

$x(t) = e^{A(t-t_0)}x_{t_0} + \int_{\tau=t_0}^{t} e^{A(t-\tau)}Bu(\tau)\,d\tau, \quad t \geq 0.$

(a) Suppose that $x(t_0) = 0$ and all eigenvalues of $A$ have their real parts as negative and $\|u\|_\infty < \infty$. Let $t_0 \to -\infty$. Show that if one is to represent $x(t) = (h * u)(t)$, we have

$h(t) = Ce^{At}B\,1_{\{t \geq 0\}}.$

(b) Alternatively, we could skip the condition that the eigenvalues of $A$ have their real parts as negative, but require that $x(0) = 0$ and $u(t) = 0$ for $t < 0$. Express the solution as a convolution $y(t) = (h * u)(t)$, and find $h(t)$.

(c) Let $y(t) = Cx(t) + Du(t)$. Repeat the above.

Exercise

Let $x(n) \in \mathbb{R}^N$ and $n \in \mathbb{Z}$. Consider a linear system given by

$x(n+1) = Ax(n) + Bu(n)$

$y(n) = Cx(n), \quad n \geq 0$

with the initial condition $x(n_0) = 0$ for some $n_0$.

(a) Suppose all the eigenvalues of $A$ are strictly inside the unit disk in the complex plane and $\|u\|_\infty < \infty$. Let $n_0 \to -\infty$. Express the solution $y(n)$ as a convolution $y(n) = (h * u)(n)$, and find that

$h(n) = CA^{n-1}B\,1_{\{n \geq 1\}}.$

(b) Alternatively, we could skip the condition that the eigenvalues of $A$ are strictly inside the unit disk in the complex plane, but require that $n_0 = 0$ so that $x(0) = 0$ and also $u(n) = 0$ for $n < 0$. Express the solution as a convolution $y(n) = (h * u)(n)$, and find $h(n)$.

(c) Let $y(n) = Cx(n) + Du(n)$. Repeat the above.

---

Bounded-Input-Bounded-Output (BIBO) Stability of Convolution Systems

---

The Frequency Response (or Transfer) Function of Linear Time-Invariant Systems

A very important property of convolution systems is that, if the input is a harmonic function, so is the output. Let

$u(t) = e^{i2\pi ft},$

be the input to a system with impulse response $h \in L_1(\mathbb{R}; \mathbb{R})$. Then,

$y(t) = \int_{\tau=-\infty}^{\infty} h(t - \tau) u(\tau)\,d\tau = \int_{\tau=-\infty}^{\infty} h(\tau) u(t - \tau)\,d\tau = \int_{\tau=-\infty}^{\infty} h(\tau) e^{i2\pi f(t-\tau)}\,d\tau,$

which leads to

$y(t) = e^{i2\pi ft} \left(\int_{-\infty}^{\infty} h(s) e^{-i2\pi fs}\,ds\right).$

Later on we will consider $u(t) = e^{st}$, $s \in \mathbb{C}$, and we will generalize the frequency response above (defined for $s = i2\pi f$, $f \in \mathbb{R}$) to the notion of transfer function of a system.

A similar discussion applies for a discrete-time system. Let $h \in l_1(\mathbb{Z}; \mathbb{R})$. If $u(n) = e^{i2\pi fn}$, then

$y(n) = \left(\sum_{m=-\infty}^{\infty} h(m) e^{-i2\pi fm}\right) e^{i2\pi fn}$

---

Steady-State vs. Transient Solutions

Let $x(t) \in \mathbb{R}^N$. Consider a system defined with the relation:

$x'(t) = Ax(t) + Bu(t), \qquad y(t) = Cx(t) + Du(t), \qquad t \geq t_0,$

for some fixed $t_0$. Consider an input $u(t) = e^{st}$, $t \geq t_0$ for some $s \in \mathbb{C}$ (for the time being, assume that $s = i2\pi f$ for some $f \in \mathbb{R}$). Suppose that $s$ is not an eigenvalue of $A$. Using the relation

$x(t) = e^{A(t-t_0)}x(t_0) + e^{At}\int_{t_0}^{t} e^{-A\tau}Be^{s\tau}\,d\tau = e^{A(t-t_0)}x(t_0) + e^{At}\int_{t_0}^{t} e^{s\tau}e^{-A\tau}B\,d\tau$

we obtain

$x(t) = \left(e^{A(t-t_0)}x(t_0) - e^{A(t-t_0)}e^{At_0}(sI - A)^{-1}e^{-At_0}e^{st_0}B\right) + \left(e^{At}(sI - A)^{-1}e^{-At}e^{st}B\right).$

Using the property that for any $t$

$e^{At}(sI - A)^{-1}e^{-At} = (sI - A)^{-1},$

we obtain

$y(t) = Ce^{A(t-t_0)}\left(x(t_0) - (sI - A)^{-1}e^{st_0}B\right) + \left(C(sI - A)^{-1}B + D\right)e^{st}, \qquad t \in \mathbb{R}_+.$

The first term is called the transient response of the system and the second term is called the steady-state response.

If $A$ is a stable matrix, with all its eigenvalues inside the unit circle, the first term decays to zero as $t$ increases (or with fixed $t$, as $t_0 \to -\infty$). Alternatively, if we set $t_0 = 0$ and write

$x(0) = (sI - A)^{-1}B,$

the output becomes

$y(t) = \left(C(sI - A)^{-1}B + D\right)e^{st}, \qquad t \geq t_0.$

The case with $s = i2\pi ft$ is crucial for stable systems. Later on we will investigate the more general case with $s \in \mathbb{C}$.

---

Bode Plots for Studying System Response to Harmonic Inputs

If we apply $u(t) = e^{i2\pi ft}$ or $e^{i\omega t}$, we observed in the above that the output would be $\hat{h}(f)e^{i2\pi ft}$.

Bode plots allow us to efficiently visualize $\hat{h}(f)$ by depicting the magnitude and phase, with a logarithmic scale; in the pre-digital era of mid-20th century in the absence of advanced computers, such plots were effective means to represent transfer functions with the logarithmic scale.

Observe that since $\hat{h}(f) = \overline{\hat{h}(-f)}$, it suffices to consider only $f \geq 0$. Let $\omega = 2\pi f$. Let $i = 1, 2, \cdots, 5$ and $s_i = r_i e^{i\theta_i}$, where $r_i = |s_i|$ and $\theta_i$ is the phase of $s_i$.

$h(i\omega) = \frac{s_1 s_2}{s_3 s_4 s_5} = \left(\frac{r_1 r_2}{r_3 r_4 r_5}\right)e^{i(\theta_1 + \theta_2 - \theta_3 - \theta_4 - \theta_5)}.$

Thus,

$|h(i\omega)| = \frac{r_1 r_2}{r_3 r_4 r_5}$

and

$\log(|h(i\omega)|) = \log(r_1) + \log(r_2) - \log(r_3) - \log(r_4) - \log(r_5).$

Note also that

$\log(e^{i(\theta_1 + \theta_2 - \theta_3 - \theta_4 - \theta_5)}) = i(\theta_1 + \theta_2 - \theta_3 - \theta_4 - \theta_5)$

so that

$\angle h(i\omega) = \theta_1 + \theta_2 - \theta_3 - \theta_4 - \theta_5.$

Thus, the logarithms allow us to consider the contributions of each complex number in an additive fashion both for the magnitude and the phase.

Building Blocks for Bode Plots

Now, consider

$h(i\omega) = K(i\omega)^n \frac{1 + i\frac{\omega}{\omega_0}}{\left(i\frac{\omega}{\omega_n}\right)^2 + 2\zeta i\frac{\omega}{\omega_n} + 1}.$

We can thus consider the contributions of $K(i\omega)^n$, $1 + i\frac{\omega}{\omega_0}$, and $\left(i\frac{\omega}{\omega_n}\right)^2 + 2\zeta i\frac{\omega}{\omega_n} + 1$ separately.

For $K(i\omega)^n$, we note that

$\log|K(i\omega)^n| = \log(|K|) + n\log(\omega)$

and

$\angle K(i\omega)^n = \angle K + n\frac{\pi}{2}.$

For $1 + i\frac{\omega}{\omega_0}$, we use the following approximations:

• For $\omega \approx 0$: $1 + i\frac{\omega}{\omega_0} \approx 1$.
• For $\omega = \omega_0$: $1 + i\frac{\omega}{\omega_0} = 1 + i$.
• For $\omega \gg \omega_0$: $|1 + i\frac{\omega}{\omega_0}| \approx \frac{|\omega|}{\omega_0}$.

Likewise, for the angle:

• For $\omega \approx 0$: $\angle\left(1 + i\frac{\omega}{\omega_0}\right) \approx 0$.
• For $\omega \gg \omega_0$: $\angle\left(1 + i\frac{\omega}{\omega_0}\right) \approx \frac{\pi}{2}$.
• At $\omega = \omega_0$: $\angle\left(1 + i\frac{\omega}{\omega_0}\right) = \frac{\pi}{4}$.

For $\left(\left(i\frac{\omega}{\omega_n}\right)^2 + 2i\zeta\frac{\omega}{\omega_n} + 1\right)^{-1}$:

• For $\omega \approx 0$, the magnitude is approximately 1, with its logarithm approximately 0.
• For $\omega = \omega_n$, the magnitude is $\frac{1}{2\zeta}$.
• For $\omega \gg \omega_n$, the magnitude decays as $-2\log(|\omega|)$.

For the phase:

• For $\omega \approx 0$, the phase is approximately 0.
• For $\omega = \omega_n$, the phase is $-\frac{\pi}{2}$.
• For $\omega \gg \omega_n$, the phase is close to $\pi$.

Bode plots approximate these expressions in a log-log plot (for the magnitude).

---

Interconnections of Systems and Feedback Control Systems

We will discuss serial connections, parallel connections, output and error feedback connections.

Control systems are those whose input-output behaviour is shaped by control laws (typically through using system outputs to generate the control inputs -- termed, output feedback --) so that desired system properties such as stability, robustness to incorrect models, robustness (to system or measurement noise) -- also called, disturbance rejection --, tracking a given reference signal, and ultimately, optimal performance are attained. These will be made precise as control theoretic applications are investigated further.

---

State-Space Description of Linear Systems

We will study state-space realizations of linear time-invariant systems in further detail in Chapter 9. We provide a brief discussion in the following.

Principle of Superposition

For a linear time-invariant system, if $(u, y)$ is an input-output pair, then $\sigma_\theta u, \sigma_\theta y$ is also an input-output pair and thus, $a_1 u + b_1 \sigma_\theta u$, $a_1 y + b_1 \sigma_\theta y$ is also such a pair.

State-Space Description of Input-Output Systems

Continuous-Time State-Space Form

Consider a continuous-time system given by:

$\sum_{k=0}^{N} a_k \frac{d^k}{dt^k} y(t) = \sum_{m=0}^{N-1} b_m \frac{d^m}{dt^m} u(t),$

with $a_N = 1$. Such a system can be written in the form:

$\frac{d}{dt}x(t) = Ax(t) + Bu(t), \qquad y(t) = Cx(t)$

$A = \begin{bmatrix} 0 & 1 & 0 & \cdots & 0 \\ 0 & 0 & 1 & \cdots & 0 \\ \vdots & \vdots & & \cdots & 0 \\ -a_0 & -a_1 & -a_2 & \cdots & -a_{N-1} \end{bmatrix}$

$B = \begin{bmatrix} 0 \\ 0 \\ \vdots \\ 0 \\ 1 \end{bmatrix}$

$C = \begin{bmatrix} b_N & b_{N-1} & \cdots & b_1 \end{bmatrix}$

Discrete-Time State-Space Form

Likewise, consider a discrete-time system of the form:

$\sum_{k=0}^{N} a_k y(n - k) = \sum_{m=1}^{N} b_m u(n - m)$

with $a_0 = 1$, can be written in the form

$x(n+1) = Ax(n) + Bu(n), \qquad y(n) = Cx(n)$

where

$x_N(n) = y(n),\; x_{N-1}(n) = y(n-1),\; \cdots,\; x_1(n) = y(n - (N-1))$

$A = \begin{bmatrix} 0 & 1 & 0 & \cdots & 0 \\ 0 & 0 & 1 & \cdots & 0 \\ \vdots & \vdots & & \cdots & 1 \\ -a_N & -a_{N-1} & -a_{N-2} & \cdots & -a_1 \end{bmatrix}$

$B = \begin{bmatrix} 0 \\ 0 \\ \vdots \\ 0 \\ 1 \end{bmatrix}$

$C = \begin{bmatrix} b_N & b_{N-1} & \cdots & b_1 \end{bmatrix}$

Stability of Linear Systems Described by State Equations

---

Exercises

Exercise

Consider a linear system described by the relation:

$y(n) = \sum_{m \in \mathbb{Z}} h(n, m)u(m), \quad n \in \mathbb{Z}$

for some $h : \mathbb{Z} \times \mathbb{Z} \to \mathbb{C}$.

(a) When is such a system causal?

(b) Show that such a system is time-invariant if and only if it is a convolution system.

Exercise

Let $x(t) \in \mathbb{R}^N$ and $t \geq 0$ and real-valued. Recall that the solution to the following differential equation:

$x'(t) = Ax(t) + Bu(t)$

with the initial condition $x(0) = x_0$ is given by

$x(t) = e^{A(t)}x_0 + \int_{\tau=0}^{t} e^{A(t-\tau)}Bu(\tau)\,d\tau, \quad t \geq t_0.$

Suppose $x(0) = 0$ and $u(t) = 0$ for $t < 0$. Express the solution as a convolution $x(t) = (h * u)(t)$, and find $h(t)$.

Note: With the assumption that the system is stable, we can avoid the condition that $u(t) = 0$ for $t < 0$. In this case, we are able to write

$x(t) = e^{At - t_0}x(t_0) + \int_{\tau=t_0}^{t} e^{A(t-\tau)}Bu(\tau)\,d\tau,$

and take the limit as $t_0 \to -\infty$, leading to $h(t) = e^{At}B\,1_{\{t \geq 0\}}$.

Exercise

Let $x(n) \in \mathbb{R}^N$ and $n \in \mathbb{Z}$. Consider a linear system given by

$x(n+1) = Ax(n) + Bu(n), \quad n \geq 0$

with the initial condition $x(0) = 0$. Suppose $x(0) = 0$ and $u(n) = 0$ for $n < 0$. Express the solution $x(n)$ as a convolution $x(n) = (h * u)(n)$, and find $h(n)$.

Note: With the assumption that the system is stable, we can avoid the condition that $u(n) = 0$ for $n < 0$. In this case, we can write

$x(n) = A^{n-n_0}x(n_0) + \sum_{m=n_0}^{n-1} A^{n-m-1}Bu(m),$

and take the limit as $n_0 \to -\infty$ leading to $h(n) = A^{n-1}B\,1_{\{n \geq 1\}}$.

Exercise

Consider a continuous-time system described by the equation:

$\frac{dy(t)}{dt} = ay(t) + u(t), \quad t \in \mathbb{R},$

where $a < 0$.

(a) Find the impulse response of the system. Is the system bounded-input-bounded-output (BIBO) stable?

(b) Suppose that the input to this system is given by $\cos(2\pi f_0 t)$. Let $y_{f_0}$ be the output of the system. Find $y_{f_0}(t)$.

(c) If exists, find

$\lim_{f_0 \to \infty} y_{f_0}(t),$

for all $t \in \mathbb{R}_+$.

Exercise

Consider a discrete-time system described by the equation:

$y(n+1) = a_1 y(n) + a_2 y(n-1) + u(n), \quad n \in \mathbb{Z}.$

(a) Is this system linear? Time-invariant?

(b) For what values of $a_1, a_2$ is the system BIBO (bounded-input-bounded-output) stable?

Exercise (Stability of Linear Time-Varying Systems)

Let $T$ be a linear system mapping with the representation;

$y(n) = \sum_{m \in \mathbb{Z}} h(n, m)u(m), \quad n \in \mathbb{Z}$

for some $h : \mathbb{Z} \times \mathbb{Z} \to \mathbb{R}$. Show that this system is BIBO stable if

$\sup_n \sum_m |h(n, m)| < \infty.$

Let us define a system to be regularly BIBO stable if for $\epsilon > 0$, $\exists \delta > 0$ such that $\|u\|_\infty \leq \delta$ implies $\|y\|_\infty \leq \epsilon$. Show that the system above is regularly BIBO stable if and only if

$\sup_n \sum_m |h(n, m)| < \infty.$

Exercise

Let $T$ be a linear system mapping $l_1(\mathbb{Z}; \mathbb{R})$ to $l_1(\mathbb{Z}; \mathbb{R})$. Show that this system is linear and continuous only if the system can be written so that with $y = T(u)$;

$y(n) = \sum_{m \in \mathbb{Z}} h(n, m)u(m), \quad n \in \mathbb{Z}$

for some $h : \mathbb{Z} \times \mathbb{Z} \to \mathbb{R}$.