The antecedent must be glass/flesh because everything else is already dust. The reference to time is "our time" and "all" has referred to people. So though there's a distant reminiscence of the concept of resurrection, I agree, the main meaning is collapse.
Dust to dust. Notice, though, that the body is a thou. Not only the soul escapes this fate, but also the mind.
There's also a problem with the logic. If the body is to collapse into anonymous dust no matter what, why would "fit" or unfit matter? In fact, the Egyptian Harper's Song as probably only the first I know to use this inevitability as excuse rather than restriction:
Lay of the Harper
'Tis well with this good prince; his day is done,
His happy fate fulfilled. So one goes forth
While others, as in days of old, remain.
The old kings slumber in their pyramids,
Likewise the noble and the learned, but some
Who builded tombs have now no place of rest,
Although their deeds were great.
Lo! I have heard the words Imhotep and Hordadaf spake
Their maxims men repeat - Where are their tombs?
Long fallen - e'en their places are unknown,
And they are now as though they ne'er had been.
No soul comes back to tell us how he fares
To soothe and comfort us ere we depart
Whither he went betimes. But let our minds
Forget of this and dwell on better things.
Revel in pleasure while your life endures
And deck your head with myrrh. Be richly clad
In white and perfumed linen; like the gods
Anointed be; and never weary grow
In eager quest of what your heart desires.
Do as it prompts you - until that sad day
Of lamentation comes, when hearts at rest
Hear not the cry of mourners at the tomb,
Which have no meaning to the silent dead.
Then celebrate this festal time, nor pause,
For no man takes his riches to the grave;
Yea, none returns again when he goes hence.
Song from the Tomb of King Intef, c. 2000 BCE
Donald Mackenzie, Egyptian Myth and Legend, pp.246f, [1907]
Notes:
Imhotep: Third dynasty official, step pyramid architect, was deified
Hordadaf: Son of the Pharaoh, Khufu, who built the largest pyramid.
Beautiful, no?!