Worth quoting Adam's Curse from Genesis:

"And to the man He said,
Because you have listened to the voice of your wife you have eaten of the tree
about which I commanded you, saying, you shall not eat from it,
cursed shall be the ground because of you;
in sorrow you shall eat of it all the days of your life.
And thorns and thistles it shall bring forth for you,
and you shall eat the plant of the field.
By the sweat of your face you shall eat bread until you return to the ground;
for out of it you have been taken;
for dust you are, and to dust you shall return. (Genesis 3:17-19)

The labor of writing poetry (and being beautiful) is the curse. Because these are not efforts with strong physical exertions, there's an irony to the title. We're in mock heroic country.

Here are the lines in which the narrator makes the mock heroic plain:

Better go down upon your marrow-bones
And scrub a kitchen pavement, or break stones
Like an old pauper, in all kinds of weather;
For to articulate sweet sounds together
Is to work harder than all these, and yet
Be thought an idler by the noisy set
Of bankers, schoolmasters, and clergymen
The martyrs call the world.

The reference to "martyrs" clinches it. Comparing being torn apart by lions to being "thought an idler" is not harsh realism. And did the martyrs think of their persecutors as men of substance without direct authority whose jobs involved doing good for people? Okay, maybe not bankers, but schoolmasters and clergymen are supposed to have benign professions.

The poet - the narrating voice - is reliably unreliable.

In the final section he writes of a private thought:

I had a thought for no one's but your ears:
That you were beautiful, and that I strove
To love you in the old high way of love;
That it had all seemed happy, and yet we'd grown
As weary-hearted as that hollow moon.

[Notice that if beauty takes work, she had been laboring as much in her way to be beautiful as he was when he "strove to love you in the old high way".]

But what had he said just above(?):

There have been lovers who thought love should be
So much compounded of high courtesy
That they would sigh and quote with learned looks
Precedents out of beautiful old books;
Yet now it seems an idle trade enough.

Idle trade is equivalent to weary-hearted. It's saying the same thing twice. Not a privately expressed sentiment at all.

I suggest that the poem is a depiction of a well-intended fool who thinks a bit too highly of himself.

In olden times (so beloved of Yeats) a journeyman was required to produce a “Masterpiece”, demonstrating mastery of technical skill and artistry. This is that. It is (I think) Yeats’ first experiment in the candid, autobiographical style of his middle life. He is beginning to feel “Responsibilities”, regret, and finding a way to art (or adulthood) only possible when an artist stops trying to make himself look cool. He's not Oisin after all.

The poet recalls a conversation several years after it occurs. He sees a young poet, full of himself, doing (almost) all the talking. He opines that art must appear effortless, but his words are not that. His description is accurate, but no example. It’s wonderful the way he works up to the line There’s many a one shall find out all heartache – hopelessly Irish/archaic, out of meter and stilted – then is answered by the young lady (in “real life” his sweetheart’s sister, but as Robert points out, converted to friend to add a frisson) – who casually, spontaneously (it seems, but she tells us it’s not true) drops a line she has thought on long and hard. She then returns to being the beautiful girl admiring the verbose poet. At the time, he barely notices. He takes her idea and runs with it, as if there were no fine thing but needs belaboring. When he figures it out, (years perhaps later) it’s too late.

This is a many-faceted poem. I’ll stop with this.

That's what's so painful. Adulthood, the end of Victorian idealism, girls making fun of one- what's a struggling Pre-Raphaelite to do?

Well, he could always imitate Gilbert's Bunthorne in Patience.

Here's what happens when he writes:

(During this Bunthorne is seen in all the agonies of composition. The Maidens are watching him intently as he writhes. At last he hits on the word he wants and writes it down. A general sense of relief.)

Bun. Finished! At last! Finished!

(He staggers, overcome with the mental strain, into the arms of Colonel.)

Colonel. Are you better now?

Bun. Yes – oh, it’s you! – I am better now. The poem is finished, and my soul has gone out into it. That was all. It was nothing worth mentioning, it occurs three times a day.

And the poem itself could be about how hollow life is in the absence of beauty and excitement:

“OH, HOLLOW! HOLLOW! HOLLOW!”

What time the poet hath hymned
The writhing maid, lithe-limbed,
Quivering on amaranthine asphodel,
How can he paint her woes,
Knowing, as well he knows,
That all can be set right with calomel?

When from the poet's plinth
The amorous colocynth
Yearns for the aloe, faint with rapturous thrills,
How can he hymn their throes
Knowing, as well he knows,
That they are only uncompounded pills?

Is it, and can it be,
Nature hath this decree,
Nothing poetic in the world shall dwell?
Or that in all her works
Something poetic lurks,
Even in colocynth and calomel?
I cannot tell.

Yes, Yeats' narrator could proclaim all of that while comfortably sitting around with people assuredly courteous.

Come to think of it, he did.

By the way, as part of the Druids' initiation process for priests, an individual was laid in a tub with water up to his nose and then a wood plank piled high with stones was placed on his chest. The tub was then covered. While waiting for his tormentors to return he had to compose a poem in a devilishly complex scheme and recite it eloquently after the stones were removed.

The poem could not be trite or lacking elevated sentiments of course. Hardship in the service of poetry should be a goad, thought the Druids, correctly.

I think that Yeats' portrait of his younger self is much more forgiving than Gilbert's picture of Oscar Wilde (though I bet he would have come off pretty well in that Druid baptism). For Yeats this poem represents a passage from youth into adulthood - as I mentioned, this is the first of his poems to really talk about his own life. He doesn't even use the word "image" once. He does not think that what he'd said was wrong, but, in retrospect and context, unreflective. Then love is mentioned.

His pain is not pretense. He's lost his place to hide.

When some people started to believe and propound the idea that Iolanthe was a political argument, Gilbert commented irritably that the opinions stated were those of the wrong-headed donkeys (characters) who said them. When Dickens' friend complained that the character based on him was a villain, Dickens responded that the character furthered his plot.

So even though the narrator might (or might not) be based on Yeats' younger self, I suggest you not consider the portrayal to be necessarily significant beyond the confines of the poem in which it appears. Though some poets write autobiographical works, they are still telling stories and inventing the characters who inhabit the plot. Including themselves.

Especially when most of the poem is Yeats' gentle joke, looking for disclosures and pain seems unlikely to be a successful effort.