if you need something temp-- i’d say do it. but for the rest of your life-- i would advise against it. hard manual labor will take its toll eventually. take a laborer job, bust your ass to impress, and ride that into an apprenticeship into a trade.
not knocking your trade–but you answered your own question. you work hard-- other unions work “smarter”. to give an example-- a laborer works harder than a carpenter, but pay does not show it.
also need to take into account the risks associated with the job. if you make a “stupid” mistake–can it hurt or kill you and coworkers? now apply that thinking to the other trades. i.e.-- electrician can be fried by one bad connection or mislabeled wire…
in the end, trades that have the most knowledge get paid more.