Your transform looks correct. To transform from world to eye coordinates, I I always use a "lookat" transform, defined by 3 vectors: $\bf{e}$, $\bf{a}$ and $\bf{u}$; in english, the eye position, the point it's looking at, and an up vector, which must not be in the same direction as $\bf{a} - \bf{e}$ (more specifically, not a multiple of it). I'm not sure how the notation given would translate, but I use these to define an orthonormal space around the eye and project into it.
The space is defined using $\bf{z} = {{(\bf{e} - \bf{a})}\over{|\bf{e} - \bf{a}}|}$, which means the negative z axis points in the direction of what I'm looking at (this works well for OpenGL); $\bf{x} = {{\bf{u} \times \bf{z}}\over{|\bf{u} \times \bf{z}|}}$ and $\bf{y} = \bf{z} \times \bf{x}$, which, again, for OpenGL, makes $\bf{x}$ rightward on the screen and $\bf{y}$ upward.
To transform into eye coordinates is a matter of subtracting they eye's position, then projecting into the space defined by the vectors above:
$$\begin{bmatrix} \bf{x}_x & \bf{x}_y & \bf{x}_z & 0 \\ \bf{y}_x & \bf{y}_y & \bf{y}_z & 0 \\ \bf{z}_x & \bf{z}_y & \bf{z}_z & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} 1 & 0 & 0 & -\bf{e}_x \\ 0 & 1 & 0 & -\bf{e}_y \\ 0 & 0 & 1 & -\bf{e}_z \\ 0 & 0 & 0 & 1 \end{bmatrix} $$
Which is exactly what you have to begin with.