Briefs to small?

Okay, I have recently bought a package of briefs because I wanted to try wearing them again after a few years. The size I bought was size small. Are briefs supose to fit really snug including tighter near the "package" where there isn't any sagging in that area of the underwear. I had bought a package of medium and they seem to fit better but look really saggy on me and are in the "package" area. I don't want them to tight because of sperm count and what, but I don't want to have a pair that are to saggy where I might as well wear boxers. Any thoughts? Any at all that will help will be appriciated thanks. (By the way, they are both collored style briefs FTL)