I swatched for a stranded color work sweater, knit the swatch in pattern, back and forth as if working in the round. I’m looking for 14 st and 17 rows to 4" and what I have after blocking is 18 st and 14 rows to 4". Now what? smaller needle and do another swatch? bigger needle?
I would go bigger needle to get stitch gauge right, and figure out how to adjust the pattern for row gauge, but others may feel differently.
I agree that achieving the stitch gauge is more important here. If it is taking more rounds to achieve 4 inches, you might consider having extra yarn available, as you’ll be knitting extra rounds.
If you purled rows it could dramatically affect gauge. Purls are usually much looser than knit stitches. If you test flat, knit all rows and just let yarn drape loosely to get back to start. The other thing that has thrown off my gauge is using different needles to swatch. Metal v wood.
As another post-er mentioned, you may get the right result by swapping type of material your needles are made of. Bamboo needles are grabbier and metal are more slippery, as you know. This starts mattering a lot in stranded work, I have found. So, change needle type, first, then size. And, it’s fair game to knit in alternating sizes.