Good behavior management involves more than eliminating bad behavior. It should also focus on developing desirable behavior. You've noticed that your horse starts to move off before you're ready, sometimes before you've even landed in the saddle. This is dangerous, because during mounting you are somewhat unbalanced, and any bad behavior on his part is likely to result in a fall on your part. So next time you get ready to get on, have the reins tight enough so that he can't walk away easily (but not so tight that you make him back up) and you can at least stop him. Then get on carefully, without poking him in the side with your foot, and after you're in the saddle, sit calmly and don't let him walk off. After 30 seconds or so, pat him, and perhaps give him a treat from the saddle, being careful not to fall forward and cause him to take a step to balance himself. Get off and do it again, again rewarding his standing still for a short period after you settle in the saddle. Gradually lengthen the time between landing on the saddle and asking him to walk off, always rewarding him for being still. It will take no more than a week for him to be standing like a rock, waiting for you to ask him to move.
Remember, behavior can be triggered by events that occur prior to the behavior (something caused your horse to look to the right and shift his weight while you were attempting to mount, knocking you off balance and making it impossible for you to mount) and are rewarded by what comes after: you led him in a circle, delaying the start of the work. And sometimes behavior just happens, as with the horse that ignored your leg aids because he was looking at something in the arena, and you failed to reinforce these aids, teaching him it's ok to ignore you.
It's important to realize that things that come before the behavior don't necessarily cause the behavior. Often they simply set the occasion for the behavior to develop, and if you are aware of this, you can take steps to prevent the possible bad behavior: for example, reinforcing his response to your leg aids by tapping him immediately with a whip if he doesn't move off promptly. You don't need to address his behavior of looking at pasture mates. So faced with a problem behavior in your horse, take some time to figure out what is going on: what happens before the behavior and what happens after-what has your response been? If the behavior seems to be getting worse, it's likely that your response is acting as a reward for the behavior, and you will have to change that situation. Try to arrange it so that your horse can be rewarded for doing what you want him to do, as in the example of standing still to be mounted.