Why optimism?
Pessimism has a lot going for it. A pessimism always sounds like a realist (pundits are often pessimistic). Pessimism avoids failure, which is often the best choice.
See for example a company like Coca-Cola or IBM or even Microsoft. Instead of being the first to experiment, they only adapt things which have been proven to work, thereby avoiding catastrophic failure. This means that at the end of the day they always survive, while other burn themselfs to the ground.
The same applies the whole continents. The optimistic US is always lamenting that the pessimistic Europe is falling, while the pessimistic Europe is always managing the adapt the things that have worked, and then steadily but persistently took over entire sectors of industry (German cars, Airbus). Slow and steady is not sexy, but it tends to survive.
The same thing applies to humans. It only takes one disaster ( this rope is surly strong enough / lets go on vacation in Somalia! ) to join the ranks of the Darwin Awards.
We tend to remember the courage adventurers who made breakthrough discoveries by flying a kit during lighting. We don’t even know the people who were electrocuted trying the same before, or maybe just as a cautionary tale ( Franz Reichelt ).
But in the longer term the optimists tend to win. Malthus was proven wrong. Not just because current constraints are not necessary future constraints, but because of the anthropic principle. The fact that we can still observe the world, proves that people that predicted the end of the world were wrong. Simply being here is proof that optimism has so far been the better choice.
But what about optimism on a personal level? What is the better choice?
Pessimism has the problem that it leads to inaction. A pessimistic person ( then call themself realists ) see only the problems.
This is not just a problem because it leads to inaction, but the bigger problem is that it is bad for the mind.
We need to believe that we can make a difference, because if we don’t we will stop trying to make a difference, which will become a self-fulfilling prophecy.
Almost al big problems or even simple problems ( I can make a better OS ) are much harder to do than a optimist thinks. But without stubbornly trying and simply ‘going for it’ and running 3x times over the budgets these projects would never have made it.
There have been so many instances of “if I would have known how difficult it was going to be I would never have started it” that validates this mindset.
Sometimes you just have to dive into a problem with minimal knowledge and to much spare time to accomplice a breakthrough. If you were a ‘realist’ you would not even have started it.
Pessimism is like like a house: you can’t live in it. The mind need a optimistic house: parts might fall down, but its better than not trying at all.