We don’t need to solve for $a$ or $b$, rather for the whole expression $a^3 + b^3$. We may need to do some factorising to put things in terms of $a+b$ and $ab$, then substitute.

It’s easy enough to isolate $a^3 + b^3$: \begin{align*} (a+b)^3 &= a^3 + 3a^2b + 3ab^2 + b^3\\ (a+b)^3 - 3a^2b - 3ab^2 &= a^3 + b^3 \end{align*} and we can substitute $5$ for $a+b$ on the lhs there, but the $-3a^2b - 3ab^2$ poses a bit of a problem. We’ll have to factorise it: \begin{align*} a^3 + b^3 &= (a+b)^3 - 3a^2b - 3ab^2\\ &= (a+b)^3 - 3ab(a + b)\\ &= 5^3 - 3\times3\times5\\ &= 125 - 45\\ &= 80. \end{align*}

That is one way to set out the solution. Another is to do the substitutions much earlier on: \begin{align*} (a+b)^3 &= a^3 + 3a^2b + 3ab^2 + b^3\\ 5^3 &= a^3 + 3a(ab) + 3(ab)b + b^3\\ 125 &= a^3 + 3a\times3 + 3\times3b + b^3\\ &= a^3 + b^3 + 9(a+b)\\ &= a^3 + b^3 + 9\times5\\ a^3 + b^3 &= 125 - 45\\ &= 80. \end{align*}