"I have seen guys in their thirties and forties still wearing college rings and T-shirts," says Iowa-born men's designer Tim Hamilton. "It's sad. I think most of it stems from laziness, but maybe there's a fear of adulthood and the commitment that goes along with it."

Like other unfortunate elements of modern men's style—pleated khakis, oversize golf shirts—college paraphernalia sticks around with the tenaciousness of a barnacle. And the widespread tendency among guys in their thirties to embrace the affectations of youth—dads with skateboards, film executives padding toward middle age in special-edition All Stars—is only making the situation worse.

"Clearly these guys are trying to express something, which in fashion isn't usually all bad," says Scott Sternberg, founder of Band of Outsiders. "But there's a fine line between nostalgia and desperation. What kills me are the Ivy League types. They're making a deliberate choice that's less about reclaiming a sense of lost youth and more about defining themselves based on a college- admissions process conducted years before."

The irony is that window-shopping in a faded Cornell T-shirt doesn't lend a man an air of success. What would lend him an air of success is doing what many of the kids currently sitting in an Econ 101 seminar—the ones who'll be interviewing with your boss in a couple of years—are doing: dressing up.

"A lot of the kids who are actually in college are more fashion-forward than the adults I see," Hamilton says. "They're dressing very sharply."

So perhaps you should put the XXL fleeces away and cultivate a polished image that suits your age and station.

If it's any comfort, know that college regalia, like plaid golf pants and fanny packs, is a sartorial affectation you can return to when you qualify for senior discounts.

"I do think it's chic when a salt-and-pepper type in his sixties is wearing an old, faded college T-shirt," Sternberg says. "But even then, it should be reserved for Sunday brunches and dog walks."