There can be multiple reasons for this:

1. Your voting power is dropping and thus your upsmokes are less influential

2. More stake is being actively used and thus the rewards are shared over more stake