When you specify a css class for your GridView’s PagerStlye (PagerStyle.CssClass=”pagerStyleClass”) the rendered table row class font-size.

You will need to select the child table of this tr element  in your css

.pagerStyleClass Table{font-size: 10px;}