Tamar,
I guess you're right on most suggestion to 'the why so late' for shale gas...
...except the last.
We basically thank the technique to an oil pioneer called George Mitchell who has been battling the technical aspects of extracting shale gas since 1981. Together with his geologists he pushed for deeper wells below his dwindling fields to tap into the source rock directly. Many indeed thought it was too costly, and it was for almost two decades, but he persevered. Now the huge potential has been unlocked and new volume estimates indicate 150 times the worlds annual gas usage being recoverable (and hopefully soon very economical).
The main problems during Mitchells quest where in the fluid and chemicals needed for fracturing of the tight rock, which proved very costly. Eventually in 1997 a mixture of water and sand proved succesful and when Devon Energy bought Mitchells company in 2002 they further enhanced the profitability by applying horizontal drilling. Profit margins are still in the margin and I guess the main advances to be made now is in adjusting day-to-day life to the use of gas. Cars, petrol stations, infrastructure, etc will have to be developed further to increase sales!
I find this a compelling story, especially as Mitchell thought out of the box and was not believed at first. It just shows huge technical discoveries can still be made and potential fame for the creative geologist is far from over...