Answer to How likely is it that governments will play a significant role? What role would be desirable, if any?
Currently, private AI labs are greatly ahead of any known governmental efforts in the production of advanced AI, so it looks unlikely that a government will be involved directly in the creation of the first AGI.
Nevertheless, governments are important drivers of policies, so in order to avoid unaligned AGI, here are some suggestions that they can enact:
- Chose policies that slow down rather than speed up AGI development, such as not incentivizing AI capabilities research.
- Attempt to foster cooperation and reduce the number of players competing to create the first AGI in order to avoid race dynamics that lead to cut corners on safety.
- Generally foster a stable, peaceful, connected world. Also don't start wars!
- How likely is it that AGI will first be developed by a large established organization, rather than a small startup, an academic group or a government? (edit question) (write answer)
- How tractable is it to get governments to play a good role (rather than a bad role) and/or to get them to play a role at all (rather than no role)? (edit question) (write answer)
- Is there something useful we can ask governments to do for AI alignment? (edit question) (write answer)