It's based exclusively on the previous year's schedule. We were number one in the division, so our 4 "uncommon" games with opponents outside the NFCE and other two divisions we all play, our games are with division winners, the Giants play the second place teams in the divisions and so on.
The Pats get to play in the ****ty AFCE against MIA, Buf and NYJ, thats why it always is ridiculously easy for them.