It may seem unfair that speculators exist, but the losers there are the previous owners and not the rest of society. For those who buy flats in a residential building, the price will be the same regardless of who gets to pocket the land appreciation, a farmer or a real estate investor.
The best way to protect the owners is to make the development process as transparent and impartial as possible, so that everybody can act on that information not just those with privileged access. For example, the local government could mandate that infrastructure development proceeds in concentric circles radiating from a virtual center point of the city, and layout a plan for a decade or two, with a number of stages. After this, money is awarded according to the plan, as the city needs to grow.
So the information about a future urban development is known a decade in advance and the true owners can enjoy a steady growth in value as the events unfold. Speculators can still invest, but they need to do so well in advance, driving the whole land appreciation by providing liquidity.
I do believe it's unfair, but my point is more about it being inefficient.
A measured and transparent expansion system would be one way to help that, although complexity would vary widely with locale (plains vs river cities).
The problem with (independent) speculators is that the city is trading a largely rational actor (agricultural land owner) with capital / a solely profit-driven actor (speculator).
My point is the latter is less inclined to compromise than the former, to the detriment of urban development.
One could make the point that aggregating speculators serve a necessary role, but I don't see much value delivered by "smaller than a planned community" types.
Furthermore, I have seen them hold up urban progress. It only takes one landowner to prevent a contiguous utility from being constructed...
Rational, in this context, means profit maximizing - both kinds of landowners should act in a similar way. If the owners compromise to less than the value of the land, sure, that's a way to redistribute the savings back to society, but it's still unfair to them. It also opens a major conflict of interest for those controlling the negotiation to get involved and grab some of that money on the table.
I much rather empower all owners to maximize the value of the land, but at the same time, have sane eminent domain regulation that prevent them for getting rich at the taxpayer's expense. So any inefficiency is the result of legislation that allows speculators to abuse the process.
Rational only means strictly profit maximizing under simple economic treatments. In this case, we're talking about improvements that boost land value, but must be agreed to / enforced on all.
Consequently, you deal with local optima, when everyone is perfectly individually greedy, no improvements can be constructed, and everything is less "good" than it could be (for the individuals and society).
The problem isn't that speculators make money per se, its when speculation becomes a feeding frenzy that jacks up the market to insane numbers. See places like Vancouver where rich foreign investors dump millions into the market to have a safe haven for their money, and don't even bother to rent it out. Why would you bother when you get double digit returns YoY by just sitting on it.
The best way to protect the owners is to make the development process as transparent and impartial as possible, so that everybody can act on that information not just those with privileged access. For example, the local government could mandate that infrastructure development proceeds in concentric circles radiating from a virtual center point of the city, and layout a plan for a decade or two, with a number of stages. After this, money is awarded according to the plan, as the city needs to grow.
So the information about a future urban development is known a decade in advance and the true owners can enjoy a steady growth in value as the events unfold. Speculators can still invest, but they need to do so well in advance, driving the whole land appreciation by providing liquidity.