Gauging a Newsboy Hat

Just started a hat that called for knitting a swatch, but rather then counting stitches per inch, it just has you knit a pattern, that when completed, is to measure 4x4. After finishing, it measured 4 inches wide, but measured 4 1/4, not counting the cast on and off rows. Should I be concerned and if so, what advice would you give. Thanks in advance.