First solve the homogeneous portion:

\(\displaystyle \\y''-8y'+16y=0 \\m^2-8m+16=0 \\(m-4)(m-4)=0\)

Therefore, \(\displaystyle m=4\) is a repeated root thus one of the complimentary solutions \(\displaystyle y_c\) is,

\(\displaystyle y_c=c_1e^{4x}+c_2xe^{4x}\)

Now find the remaining complimentary solution \(\displaystyle y_p\).

\(\displaystyle \\y_p=Ax+B \\y_p'=A \\y_p''=0\)

Now solve for \(\displaystyle A\) and \(\displaystyle B\).

\(\displaystyle \\0-8A+16(Ax+B)=24x+2 \\-8A+16Ax+16B=24x+2\)

Where 

\(\displaystyle \\16A=24 \\\\A=\frac{24}{16}=\frac{3}{2}\)

and 

\(\displaystyle \\-8A+16B=2 \\\\-8\left(\frac{3}{2} \right )+16B=2 \\\\-\frac{24}{2}+16B=2 \\\\-12+16B=2 \\\\16B=14 \\\\B=\frac{14}{16}=\frac{7}{8}\)

Therefore,

\(\displaystyle y_p=\frac{3}{2}x+\frac{7}{8}\)

Now, combine both of the complimentary solutions together to arrive at the general solution.

\(\displaystyle \\y=y_c+y_p \\y=c_1e^{4x}+c_2xe^{4x}+\frac{3}{2}x+\frac{7}{8}\)

This is a higher order inhomogeneous linear differential equation. Because the inhomogeneity is a cosine, we will use variation of parameters to solve it.

First, we find the characteristic equation to solve for the homogenous solution. This gives us \(\displaystyle \lambda^2 - 3\lambda - 4 = (\lambda -4)(\lambda + 1) = 0\).

This tells us that the homogeneous solution is \(\displaystyle c_1e^{4t} + c_2e^{-t}\). As neither of these overlap with our inhomogeneity, we are safe to continue without adding a factor of t.

Thus, let us guess that \(\displaystyle y_p = a_1\sin(2t) + a_2\cos(2t)\). Then,

\(\displaystyle y_p' = 4a_1\cos(2t) - 4a_2\sin(2t)\) and

\(\displaystyle y_p'' = -4a_1\sin(2t) - 4a_2\cos(2t)\)

Plugging into the original equation, we have

\(\displaystyle \sin(2t)(-4a_1 + 6a_2 -4a_1) + \cos(2t)(-4a_2 - 6a_1 -4a_2) = -25\cos(2t)\)

Which implies that \(\displaystyle -8a_1 + 6a_2 = 0\) and \(\displaystyle -8a_2 - 6a_1 = -25\). Solving via substitution,

 \(\displaystyle a_1 = \frac{3}{4}a_2\)

\(\displaystyle -8a_2 - \frac{9}{2}a_2 = -\frac{25}{2}a_2 = -25; a_2 = 2\)

\(\displaystyle a_1 = \frac{3}{2}\)

Thus, the particular solution is \(\displaystyle y_p = \frac{3}{2}\sin(2t) + 2\cos(2t)\) and the overall solution is the particular plus the homogeneous.

So 

\(\displaystyle y = c_1e^{4t} + c_2e^{-t} + \frac{3}{2}\sin(3t) + 2\cos(3t)\)

We first note that the differential equation has characteristic equation 

\(\displaystyle r^2+3=0\) ,

since the roots of this characteristic polynomial are linearly independent of the forcing function   

\(\displaystyle t^{2}e^{2t}\),

we simply use undetermined coefficient combination rules to figure that the particular solution will be of form 

\(\displaystyle e^{2t}(At^2+Bt+C)\)

We first figure that the forcing function  \(\displaystyle t^2sin(2t)\) is linearly independent  to the homogeneous solution solved with the characteristic equation.

Therefore, using proper undetermined coefficients function rules, the particular solution will be of the form:

\(\displaystyle y_{p} = (At^2+Bt+C)sin(2t) + (Dt^2+Et+F)cos(2t)\) 

It is important to note that when either a sine or a cosine is used, both sine and cosine must show up in the particular solution guess.

We start with the assumption that the particular solution must be of the form 

\(\displaystyle y_{p} =Ae^{3t}\) .

Then we solve the first and second derivatives with this assumption, that is, 

\(\displaystyle y_p'=3Ae^{3t}\) and \(\displaystyle y_p''=9Ae^{3t}\).

Then we plug in these quantities into the given equation to get:

\(\displaystyle 9Ae^{3t}-6Ae^{3t} +5Ae^{3t} = 4Ae^{3t}\) , which solves for \(\displaystyle A = \frac{1}{2}\).

Thus, but the method of undetermined coefficients, a particular solution to this differential equation is:

\(\displaystyle y_p=\frac{e^{3t}}{2}\)

First solve the homogeneous portion:

\(\displaystyle \\y''-8y'+16y=0 \\m^2-8m+16=0 \\(m-4)(m-4)=0\)

Therefore, \(\displaystyle m=4\) is a repeated root thus one of the complimentary solutions \(\displaystyle y_c\) is,

\(\displaystyle y_c=c_1e^{4x}+c_2xe^{4x}\)

Now find the remaining complimentary solution \(\displaystyle y_p\).

\(\displaystyle \\y_p=Ax+B \\y_p'=A \\y_p''=0\)

Now solve for \(\displaystyle A\) and \(\displaystyle B\).

\(\displaystyle \\0-8A+16(Ax+B)=24x+2 \\-8A+16Ax+16B=24x+2\)

Where 

\(\displaystyle \\16A=24 \\\\A=\frac{24}{16}=\frac{3}{2}\)

and 

\(\displaystyle \\-8A+16B=2 \\\\-8\left(\frac{3}{2} \right )+16B=2 \\\\-\frac{24}{2}+16B=2 \\\\-12+16B=2 \\\\16B=14 \\\\B=\frac{14}{16}=\frac{7}{8}\)

Therefore,

\(\displaystyle y_p=\frac{3}{2}x+\frac{7}{8}\)

Now, combine both of the complimentary solutions together to arrive at the general solution.

\(\displaystyle \\y=y_c+y_p \\y=c_1e^{4x}+c_2xe^{4x}+\frac{3}{2}x+\frac{7}{8}\)

The form of a guess for a particular solution is 

\(\displaystyle (A_3t^4+A_2t^3+A_1t^2+A_0t)\cos{3t} + (B_3t^4+B_2t^3+B_1t^2+B_0t)\sin{3t}\)