I've been in your shoes, and I know how crucial it is to pick the right monitors for mixing and mastering. Both the Yamaha HS8 and KRK Rokit series are popular choices, and each has its strengths. The Yamaha HS8 is well-regarded for its flat frequency response, which is essential for critical listening tasks. It tends to reveal more detail and honesty in the mix, although some users find the high frequencies a bit fatiguing over long sessions.
On the other hand, the KRK Rokit series, especially the newer generations, have a reputation for a more colored sound, particularly in the low end. This can be beneficial if you're working on bass-heavy genres, but it might not provide the neutrality you're seeking for mastering.
In smaller studio spaces, both models can work effectively, but positioning is key. Ideally, you want to place your monitors so that they form an equilateral triangle with your listening position, ensuring they're at ear level. Also, consider acoustic treatment, especially behind the speakers and at the first reflection points on the walls, to mitigate room-related issues.
As for features, look for monitors with room correction capabilities or switchable EQ settings if your space is less than ideal. This can help tailor the sound to your environment somewhat.
You might also want to check some comparison reviews or even try to audition them somewhere, because personal preference plays a big role. Audio world experts like Sound On Sound and published guides from mastering engineers can offer deeper insights into specific models. What genre do you mainly work with? That could also influence which monitor might suit your needs best.