I think that I see what you mean. For me, the correct "change of basis", going from $(x,y)$ to $(u,v)$ does not take into account a "translation" of the basis into another: the origin of the basis is ignored (it is not even in the definition of what a basis is). So the second paragraph is the most accurate mathematically speaking: the only thing that matters is the rotation between the basis vectors (assuming all basis vectors have the same length).
However, in practice in Graphics, it is more convenient to attach different basis to different origins points. To account for that, you now need the first formula, which is a change of basis plus a translation:
$$\begin{bmatrix}
x_p\\
y_p
\end{bmatrix}
= \begin{bmatrix}
x_u& x_v \\
y_u& y_v \\
\end{bmatrix}
\begin{bmatrix}
u_p\\v_p
\end{bmatrix} +
\begin{bmatrix}
x_e\\y_e
\end{bmatrix}
= R \begin{bmatrix}
u_p\\v_p
\end{bmatrix} +
\begin{bmatrix}
x_e\\y_e
\end{bmatrix}$$
This writting is quite heavy, so an additional component is usually used to add translation transformations, and that leads to the formula that is in the book:
$$\begin{bmatrix}
x_p\\
y_p\\
1
\end{bmatrix}
= \begin{bmatrix}
x_u& x_v&x_e \\
y_u& y_v&y_e \\
0&0&1\\
\end{bmatrix}
\begin{bmatrix}
u_p\\v_p\\1
\end{bmatrix}$$
(For 3D basis, it would lead to matrices with 4x4 components.)
I hope that it helps?
[edit]
Also, the confustion can come from the fact that the matrix R can be seen in two ways: a tool to express the coordinates in another basis, or a transformation matrix (a rotation). Therefore, if you use this time the transpose of the $R$ matrix that I have written earlier:
$$R_{uv}=\begin{bmatrix}
x_u&y_u \\
x_v& y_v \\
\end{bmatrix} $$ this matrix can express the coordinates from (xy) frame to the (uv) frame:
$$\begin{bmatrix} u_u\\ v_u\end{bmatrix}= \begin{bmatrix} x_u&y_u \\
x_v& y_v \\ \end{bmatrix}\begin{bmatrix} x_u\\y_u\end{bmatrix} =\begin{bmatrix}
1\\0
\end{bmatrix}$$
but it can also be used as a tranformation in the (xy) frame that would rotate the vector u to the vector x, and the vector v to the vector y
$$\begin{bmatrix} x_{x}\\ y_{x}\end{bmatrix}= \begin{bmatrix} x_u&y_u \\
x_v& y_v \\ \end{bmatrix}\begin{bmatrix} x_u\\y_u\end{bmatrix}$$
$$
\mathbf{x}=R\,\mathbf{u}
$$
