The same top tier schools claim to value diversity very much. Are you going to do better? If you decide to pull from another college at random to counteract only going to the same top five, chances are that college doesn't value diversity as much, and you'll get a less diverse candidate.

If you're focusing on good hires and not necessarily diverse ones, then yeah it's not smart to aggressively filter by school. Talent (even amazing talent) is not exclusive to any member of such a broadly defined set classification.