The danger is not some future system that might generate perfect prose, the danger is a current system that creates credible bullshit for a fraction of a cent. Because the effects I described in the previous paragraph are no accidents, they are what these systems are for. While OpenAI is a cult, Microsoft doesn’t care about statistically producing the singularity: They want to give people who pay them tools that make them more cost-effective. That optimize the use of capital. Putting pressure on labour is not an accident but that narrative is the most important thing about the whole “AI” discourse: “Don’t ask for fair wages or we’ll automate you”.
The open letter addresses none of that. It’s wishful worries of a group of people who read way too much science fiction and way too little about the political economy and structures of reality.
If you care about what these existing systems do, how they can cause harm, ask for a ban. Make their use illegal. Tax them so heavily that they become economically invalid. Do something that doesn’t just say “okay, GPT-4 can stay with all the harms it does but nobody else can work on it because OpenAI’s/Microsoft’s monopoly is really neat”.