Matthew 22:11-12 — And when the king came in to see the guests, he saw there a man which had not on a wedding garment: and he saith unto him, Friend, how camest thou in hither not having a wedding garment? And he was speechless.

How much does it really matter? The way we dress when we go to the house of The Lord? I know that when I was a young man there was a very well understood “dress code”. Everybody wore their “Sunday Best”! I know that when I look about me in most churches today, things are very different. “Sunday Best” is an idea that has had its day. (It was a pretty new idea anyway — for throughout history most people could never have afforded a special Sunday suit anyway).
So, how much does it really matter? At a first glance at my key verses, it seems as though it might. But we should look closer at the intention of the parable. What was it about this man that was so objectionable? Alexander MacLaren is one of many commentators to make the matter clear:

What, then, is the wedding garment? It can be nothing else than righteousness, moral purity, which fits for sitting at His table in His kingdom. And the man who has it not, is the nominal Christian, who says that he has accepted God’s invitation, and lives in sin, not putting off ‘the old man with his deeds,’ nor putting on ‘the new man, which is created in righteousness.’

There are, to be sure, good reasons to have a care how we dress for church. To dress in a way that in any way promotes ourselves, or distracts others, must be wrong. Yet beyond this, it is not so great a matter. Peter and Paul both (and I’m not sure either one was any kind of fashion model) commented on the relative importance of outward appearance and inward reality.
Peter, talking about the proper behavior of wives said:

Whose adorning, let it not be that outward adorning of plaiting the hair, and of wearing of gold, or of putting on of apparel; but let it be the hidden man of the heart, in that which is not corruptible, even the ornament of a meek and quiet spirit, which is in the sight of God of great price. (1 Peter 3:3-4)

Paul, for his part said:

I will therefore that men pray every where, lifting up holy hands, without wrath and doubting. In like manner also, that women adorn themselves in modest apparel, with shamefacedness and sobriety; not with braided hair, or gold, or pearls, or costly array; but (which becometh women professing godliness) with good works. (1 Timothy 2:8-10)

Both Peter and Paul were writing to specific audiences at a particular time. What they said was addressed, especially, to the issue of “better off” women in the first century Christian church queening it over their poorer sisters. But those same comments could be addressed with equal force to today’s churchgoers, men and women. Anyone who pays more attention to how people are dressed than to their attitudes and activities is completely missing the point.
So go ahead. If you feel it’s God-honoring, wear your Sunday Best. But don’t make it a point of difference between you and those around you in church.