DEFINITELY!!
Just do it. You really need to see this place, as well as the Stockmans Hall Of Fame across the road. Oh, and the old Powerhouse Museum around one of the back streets while you're at it.
If the Qantas museum and Hall Of Fame were down at the coast, you couldn't move for the visitors. As it is, they're pretty busy.
I rode out there last year on my bike...it's about 500km from here to Longreach, and was a pleasant ride on the bike leaving before dawn and arriving at dinnertime.
The photos above are far better than any I have, so they pretty much sum it up. If you have kids they will love it!
I'll agree that you should also set some time aside to check out Ilfracombe on the way out...the lf machinery there is astounding, as well as the amazing (and free!) bottle museum in the center of it all...thousands of bottles of all shapes and sizes are there on display.
One sour note is in Barcaldine as you head out to Longreach...you'll see the monstrosity long before you get to Barcy...a massive 18 meter square monument to a political party, the "Tree Of Knowledge" memorial...
This ugly blight on the landscape was built right in the middle of town, at a cost of EIGHT. MILLION. FRICKING. TAXPAYERS. DOLLARS! Imagine how much that money could have helped the Flying Doctor, local hospitals, or other essential services instead of chest-beating for one damn political party!

This is the underside of it, with the stump of the tree where it has been moved. Interestingly, those hanging lumps of wood are about four inches square and several meters long...each hanging on a single bolt. They're already cracking, and some nuts have come loose. In the memorable words of one disgruntled old guy in a pub who spoke to us about them, "If one of them fell and hit you on the head, it wouldn't stop until it came out your bum..."
More sickening, just an hour down the road at the same time this enormous money pit was being built, the little town of Blackall was trying to build a statue of a First World War hero who had been a local kid, and who won the Victoria Cross for bravery...while the government was throwing many millions of taxpayers dollars at the great black cube up the road in Barcaldine, they had to have raffles and beg for money to raise the $30,000 or so to put up a statue to this hero...
Back to the subject...go to Longreach...you won't regret it. I stayed in a caravan park, but there are great hotels and cabins available elsewhere, depending on what you're after. A quick web search will turn up all you need.
Oh, and go to the Longreach RSL for dinner...try the reef and beef...you won't regret it either...
And if you're driving out from Rockhampton, stop at Alpha at the bakery in the middle of what passes for the main street...Snows Bakery I think, an award winning place. The sausage rolls will have you considering selling up and moving to Alpha to live just so you can have one of them every day...
EDIT: Just found some pictures from my ride out there last year...
Pie and snag roll in Alpha for breakfast...
My GSX-1400 in front of the Stockmans Hall of Fame...
Reef & Beef at the Longreach RSL...
The looooooong straights from Alpha onwards to the west...and yes, I only did only 110kph...officer...