In Sapiens, on p. 87 of my copy:
Then why didn't humans abandon farming when the plan backfired? Partly because it took generations for the small changes to accumulate and transform society and, by then, nobody remembered that they had ever lived differently. And partly because population growth burned humanity's boats. If the adoption of ploughing increased a village's population from a hundred to 110, which ten people would have volunteered to starve so that the others could go back to the good old times? There was no going back. The trap snapped shut.
The pursuit of an easier life resulted in much hardship, and not for the last time. It happens to us today. How many young college graduates have taken demanding jobs in high-powered firms, vowing that they will work hard to earn money that will enable them to retire and pursue their real interests when they are thirty-five? But by the time they reach that age, they have large mortgages, children to school, houses in the suburbs that necessitate at least two cars per family, and a sense that life is not worth living without really good wine and expensive holidays abroad. What are they supposed to do, go back to digging up roots? No, they double their efforts and keep slaving away.