I usually join people like Elon Musk, Isaac Asimov and many others who genuinely fear AI. IMO if we invent a hyper-optimizing self-developing machine we’re most likely toast and will be used for bio-fuel without a second thought by the AI. It will likely view us as inferior / previous-generation bio-machines that need to be removed so the ecosystem can thrive (sadly they might even be correct, having in mind that we keep causing climate change) and to facilitate the emergence of next-gen machines.
That being said, what we actually should fear much more, and fear it today, is the murder of all creativity by mercilessly shovelling potentially creative minds into menial and bullsh*t jobs.
Us the techies can still make a huge difference and improve the world vastly but (a) there is a lot of politics, inertia and vested interest preventing us from doing so and (b) we actually have to have the free time, energy and motivation to work on these things. And the sad reality is that we want to care for our own health and well-being (and that of our families). Between that and work and some leisure time, a separate free time for creative and world-changing endeavours seem to be a pipe dream…
So I’d say fear of AI is hugely overrated. Even with the relentless personal data harvesting, there’s a proof emerging that the efficiency of the manipulation by well-targeted ads is slowly diminishing.
IMO the current state of affairs is – we are very quickly speeding to a head-first crash into the wall named “we will be completely stuck technologically for a long time unless the financial incentive systems change radically”.