you're wasting a lot of polygons, especially on the legs. Usually, an 8 sided cylinder is enough, even pushing it. And try to add some loops to the shoulders, for better deformation.
check polycount, there are some skinning vids there, don't know if they're still available. Also, poopinmymouth.com has an excellent vid tut on skinning.