When I plug in my headphones you can still hear the noise so it is probably not the problem with the speakers?
It just has a bad sound chip, like most PC's. Only a USB audio interface can help. Favorably under 40 e the Behringer UMC22. Slight residual noise. Good from approx. 100 euro eg from Steinberg or Tascam or Focusrite