Edit Question: Which organizations are working on AI policy?

You do not have permission to edit this page, for the following reason:

Question:Sets the destination title to move this page and attached answers to.
Alternate Phrasings:Alternate ways of phrasing this question to improve search.
Tags:	AcademiaAcausal tradeActorsAgencyAgent foundationsAgiAgi fire alarmAiAi safety campAi safety supportAi takeoffAi takeoverAlgorithmic progressAlignmentAlignment proposalsAlignment targetsAnthropicArcArchitecturesArms raceAsiAutomationAutonomous weaponsAwarenessBenefitsBiological cognitive enhancementBoxingBrainBrain-computer interfacesCapabilitiesCareersCenter for human compatible ai (chai)CivilizationClrCognitive enhancementCognitive superpowersCoherent extrapolated volitionCollaborationCommunicationCommunityComplexity of valueComprehensive ai servicesComputeComputing overhangConjectureConsciousnessContentContributingControl problemCooperative inverse reinforcement learning (cirl)CorrigibilityCounterfactualsCreativityCybersecurityCyborgismDebateDeceptionDeceptive alignmentDecision theoryDeep learningDeepmindDefinitionsDeveloppersDifferential technological developmentDifficulty of alignmentDo what i meanDoomEducationEliezer yudkowskyElkElon muskEmbedded agencyEmotionsEnculturedEpistomologyEric drexlerEthicsEutopiaEvolutionExistential riskFarFormal proofFriendly aiFundingFuture of humanity instituteGame theoryGoalsGoodhart's lawGovernanceGovernmentGptGpt-3HedoniumHuman valuesHuman-in-the-loopHumansImpact measuresImplementationIncentivesInfohazardInformation securityInfra-bayesianismInfrastructureInner alignmentInside viewInstitutionsInstrumental convergenceIntelligenceIntelligence amplificationIntelligence explosionInterpretabilityInvestmentingLanguage modelsLiteratureMachine learningMaximizersMegaprojectsMentorshipMesa-optimizationMetaethicsMetaphilosophyMetaphorsMiriMolecular nanotechnologyMotivationMultipolarMyopiaNarrow aiNatural languageNearest unblocked strategyNeeds workNeural networksNeuromorphic aiNick bostromNoneObjectionsOpen problemOpenaiOrganizationsOrthogonality thesisOther causesOughtOutdatedOuter alignmentOutreachPaperclip maximizerParadigmPattern-matchingPeoplePerson-affecting viewPersonal actionPersuasionPhilosophyPhysicsPlausibilityPlex's answer to what are some good resources on ai alignment?PolicyPoliticsPower seekingPreferencesProductivityProgrammingPsychologyQuantilizersRace dynamicsRay kurzweilRecursive self-improvementRedwood researchRegulationResearch agendasResearch assistantsResourcesRobin hansonRobotsRobustnessS-riskScaling lawsSecretSecurity mindsetSeed aiShard theorySimulation hypothesisSingletonSolutionsSpecification gamingStable win conditionStampyStop buttonStubStudySuccess modelsSuperintelligenceSurveysTech companiesTechnological unemploymentTechnologyTestTest tagTimelinesTool aiToolingTransformative aiTranshumanismTreacherous turnTripwireTruthful aiUtility functionsValue learningValuesWhat aboutWhole brain emulationWhy not justWireheadinggovernance
Related questions:	A lot of concern appears to focus on human-level or “superintelligent” AI. Is that a realistic prospect in the foreseeable future?AIs aren’t as smart as rats, let alone humans. Isn’t it far too early to be worrying about this kind of thing?Any AI will be a computer program. Why wouldn't it just do what it's programmed to do?Are AI researchers trying to make conscious AI?Are Google, OpenAI, etc. aware of the risk?Are any major politicians concerned about this?Are expert surveys on AI safety available?Are there any AI alignment projects which governments could usefully put a very large amount of resources into?Are there any plausibly workable proposals for regulating or banning dangerous AI research?Are there promising ways to make AI alignment researchers smarter?Are there risk analysis methods, which may help to make the risk more quantifiable or clear?Are there types of advanced AI that would be safer than others?Aren't robots the real problem? How can AI cause harm if it has no ability to directly manipulate the physical world?Aren’t there some pretty easy ways to eliminate these potential problems?At a high level, what is the challenge of alignment that we must meet to secure a good future?Can AI be creative?Can an AI really be smarter than humans?Can humans stay in control of the world if human- or superhuman-level AI is developed?Can people contribute to alignment by using proof assistants to generate formal proofs?Can we add "friendliness" to any artificial intelligence design?Can we constrain a goal-directed AI using specified rules?Can we ever be sure that an AI is aligned?Can we get AGI by scaling up architectures similar to current ones, or are we missing key insights?Can we program the superintelligence to maximize human pleasure or satisfaction of human desires?Can we teach a superintelligence a moral code with machine learning?Can we tell an AI just to figure out what we want and then do that?Can we test an AI to make sure that it’s not going to take over and do harmful things after it achieves superintelligence?Can you give an AI a goal which involves “minimally impacting the world”?Can you stop an advanced AI from upgrading itself?Can't we just tell an AI to do what we want?Can’t we just program the superintelligence not to harm us?Considering how hard it is to predict the future, why do we think we can say anything useful about AGI today?Could AI have basic emotions?Could I contribute by offering coaching to alignment researchers? If so, how would I go about this?Could an AGI have already been created and currently be affecting the world?Could divesting from AI companies without good safety culture be useful, or would this be likely to have a negligible impact?Could emulated minds do AI alignment research?Could we build provably beneficial AI systems?Could we get significant biological intelligence enhancements long before AGI?Could we program an AI to automatically shut down if it starts doing things we don’t want it to?Could we tell the AI to do what's morally right?Could weak AI systems help with alignment research?Do you need a PhD to work on AI Safety?Does it make sense to focus on scenarios where change is rapid and due to a single actor, or slower and dependent on getting agreements between several relevant actors?Does the importance of AI risk depend on caring about transhumanist utopias?Even if we are rationally convinced about the urgency of existential AI risk, it can be hard to feel that emotionally because the danger is so abstract. How can this gap be bridged?How can I be a more productive student/researcher?How can I collect questions for Stampy?How can I contact the Stampy team?How can I contribute in the area of community building?How can I contribute to Stampy?How can I convince others and present the arguments well?How can I get hired by an organization working on AI alignment?How can I join the Stampy dev team?How can I support alignment researchers to be more productive?How can we interpret what all the neurons mean?How close do AI experts think we are to creating superintelligence?How could an intelligence explosion be useful?How could general intelligence be programmed into a machine?How could poorly defined goals lead to such negative outcomes?How difficult should we expect alignment to be?How do I add content from LessWrong / Effective Altruism Forum tag-wikis to Stampy?How do I form my own views about AI safety?How do I format answers on Stampy?How do I know whether I'm a good fit for work on AI safety?How do I stay motivated and productive?How do I stay updated about AI progress?How do organizations do adversarial training and red teaming?How do the incentives in markets increase AI risk?How does AI taking things literally contribute to alignment being hard?How does MIRI communicate their view on alignment?How does the current global microchip supply chain work, and who has political power over it?How does the stamp eigenkarma system work?How doomed is humanity?How fast will AI takeoff be?How good is the world model of GPT-3?How hard is it for an AGI to develop powerful nanotechnology?How important is research closure and OPSEC for capabilities-synergistic ideas?How is "intelligence" defined?How is AGI different from current AI?How is Beth Barnes evaluating LM power seeking?How is OpenAI planning to solve the full alignment problem?How is metaethics relevant to AI alignment?How is the Alignment Research Center (ARC) trying to solve Eliciting Latent Knowledge (ELK)?How likely are AI organizations to respond appropriately to the risks of their creations?How likely is an "intelligence explosion"?How likely is it that AGI will first be developed by a large established organization, rather than a small startup, an academic group or a government?How likely is it that an AI would pretend to be a human to further its goals?How likely is it that governments will play a significant role? What role would be desirable, if any?How long will it be until superintelligent AI is created?How might "acausal trade" affect alignment?How might AGI interface with cybersecurity?How might AGI kill people?How might a real-world AI system that receives orders in natural language and does what you mean look?How might a superintelligence socially manipulate humans?How might a superintelligence technologically manipulate humans?How might an "intelligence explosion" be dangerous?How might an AI achieve a seemingly beneficial goal via inappropriate means?How might non-agentic GPT-style AI cause an "intelligence explosion" or otherwise contribute to existential risk?How might things go wrong with AI even without an agentic superintelligence?How might we get from Artificial General Intelligence to a Superintelligent system?How might we reduce the chance of an AI arms race?How might we reduce the diffusion of dangerous AI technology to insufficiently careful actors?How much can we learn about AI with interpretability tools?How much resources did the processes of biological evolution use to evolve intelligent creatures?How possible (and how desirable) is it to change which path humanity follows to get to AGI?How powerful will a mature superintelligence be?How quickly could an AI go from the first indications of problems to an unrecoverable disaster?How quickly would the AI capabilities ecosystem adopt promising new advances in AI alignment?How should I change my financial investments in response to the possibility of transformative AI?How should I decide which quality level to attribute to a proposed question?How should I personally prepare for when transformative AI arrives?How software- and/or hardware-bottlenecked are we on AGI?How successfully have institutions managed risks from novel technology in the past?How tractable is it to get governments to play a good role (rather than a bad role) and/or to get them to play a role at all (rather than no role)?How would I know if AGI were imminent?How would we align an AGI whose learning algorithms / cognition look like human brains?How would we know if an AI were suffering?How would you explain the theory of Infra-Bayesianism?I want to help out AI alignment without necessarily making major life changes. What are some simple things I can do to contribute?I want to work on AI alignment. How can I get funding?I'm interested in working on AI safety. What should I do?If AGI comes from a new paradigm, how likely is it to arise late in the paradigm when it is already deployed at scale, versus early on when only a few people are exploring the idea?If AI takes over the world how could it create and maintain the infrastructure that humans currently provide?If I only care about helping people alive today, does AI safety still matter?If an AI became conscious, how would we ever know?If we solve alignment, are we sure of a good future?In "aligning AI with human values", which humans' values are we talking about?In what ways are real-world machine learning systems different from expected utility maximizers?Is AI alignment possible?Is AI safety research racing against capability research? If so, how can safety research get ahead?Is expecting large returns from AI self-improvement just following an exponential trend line off a cliff?Is it already too late to work on AI alignment?Is it likely that hardware will allow an exponential takeoff?Is it possible to block an AI from doing certain things on the Internet?Is it possible to code into an AI to avoid all the ways a given task could go wrong, and would it be dangerous to try that?Is large-scale automated AI persuasion and propaganda a serious concern?Is merging with AI through brain-computer interfaces a potential solution to safety problems?Is the UN concerned about existential risk from AI?Is the focus on the existential threat of superintelligent AI diverting too much attention from more pressing debates about AI in surveillance and the battlefield, and its potential effects on the economy?Is the question of whether we're living in a simulation relevant to AI safety? If so, how?Is there a Chinese AI safety community? Are there safety researchers working at leading Chinese AI labs?Is there a danger in anthropomorphizing AI’s and trying to understand them in human terms?Is there something useful we can ask governments to do for AI alignment?Is this about AI systems becoming malevolent or conscious and turning on us?Isn't it hard to make a significant difference as a person who isn't going to be a world-class researcher?Isn't it too soon to be working on AGI safety?Isn't the real concern AI being misused by terrorists or other bad actors?Isn't the real concern AI-enabled totalitarianism?Isn't the real concern autonomous weapons?Isn't the real concern technological unemployment?Isn’t AI just a tool like any other? Won’t it just do what we tell it to?Isn’t it immoral to control and impose our values on AI?I’d like to get deeper into the AI alignment literature. Where should I look?Might an "intelligence explosion" never occur?Might an aligned superintelligence force people to "upload" themselves, so as to more efficiently use the matter of their bodies?Might an aligned superintelligence force people to have better lives and change more quickly than they want?Might an aligned superintelligence immediately kill everyone and then go on to create a "hedonium shockwave"?Might attempting to align AI cause a "near miss" which results in a much worse outcome?Might humanity create astronomical amounts of suffering when colonizing the universe after creating an aligned superintelligence?Might trying to build a hedonium-maximizing AI be easier and more likely to work than trying for eudaimonia?OK, I’m convinced. How can I help?Once we notice that a superintelligence given a specific task is trying to take over the world, can’t we turn it off, reprogram it or otherwise correct the problem?Should I engage in political or collective action like signing petitions or sending letters to politicians?Should we expect "warning shots" before an unrecoverable catastrophe?Superintelligence sounds like science fiction. Do people think about this in the real world?This all seems rather abstract. Isn't promoting love, wisdom, altruism or rationality more important?To what extent are there meaningfully different paths to AGI, versus just one path?We already have psychopaths who are "misaligned" with the rest of humanity, but somehow we deal with them. Can't we do something similar with AI?We’re going to merge with the machines so this will never be a problem, right?What about AI concerns other than misalignment?What about having a human supervisor who must approve all the AI's decisions before executing them?What actions can I take in under five minutes to contribute to the cause of AI safety?What alignment strategies are scalably safe and competitive?What approaches are AI alignment organizations working on?What are "coherence theorems" and what do they tell us about AI?What are "human values"?What are "scaling laws" and how are they relevant to safety?What are "selection theorems" and can they tell us anything useful about the likely shape of AGI systems?What are Encultured working on?What are OpenAI Codex and GitHub Copilot?What are Scott Garrabrant and Abram Demski working on?What are alternate phrasings for?What are brain-computer interfaces?What are language models?What are likely to be the first transformative applications of AI?What are mesa-optimizers?What are plausible candidates for "pivotal acts"?What are some AI alignment research agendas currently being pursued?What are some good books about AGI safety?What are some good podcasts about AI alignment?What are some good resources on AI alignment?What are some helpful AI policy ideas?What are some important examples of specialised terminology in AI alignment?What are some objections to the importance of AI alignment?What are some of the leading AI capabilities organizations?What are some of the most impressive recent advances in AI capabilities?What are some open research questions in AI alignment?What are some practice or entry-level problems for getting into alignment research?What are some problems in philosophy that are related to AI safety?What are some specific open tasks on Stampy?What are the "win conditions"/problems that need to be solved?What are the differences between AGI, transformative AI and superintelligence?What are the differences between “AI safety”, “AGI safety”, “AI alignment” and “AI existential safety”?What are the different possible AI takeoff speeds?What are the different versions of decision theory?What are the editorial protocols for Stampy questions and answers?What are the leading theories in moral philosophy and which of them might be technically the easiest to encode into an AI?What are the potential benefits of AI as it grows increasingly sophisticated?What are the style guidelines for writing for Stampy?What assets need to be protected by/from the AI? Are "human values" sufficient for it?What beneficial things would an aligned superintelligence be able to do?What can I do to contribute to AI safety?What can we expect the motivations of a superintelligent machine to be?What convinced people working on AI alignment that it was worth spending their time on this cause?What could a superintelligent AI do, and what would be physically impossible even for it?What does Elon Musk think about AI safety?What does Evan Hubinger think of Deception + Inner Alignment?What does MIRI think about technical alignment?What does Ought aim to do?What does a typical work day in the life of an AI safety researcher look like?What does alignment failure look like?What does generative visualization look like in reinforcement learning?What does the scheme Externalized Reasoning Oversight involve?What evidence do experts usually base their timeline predictions on?What external content would be useful to the Stampy project?What harm could a single superintelligence do when it took so many humans to build civilization?What if technological progress stagnates and we never achieve AGI?What if we put the AI in a box and have a second, more powerful, AI with the goal of preventing the first one from escaping?What is "Do What I Mean"?What is "HCH"?What is "agent foundations"?What is "biological cognitive enhancement"?What is "coherent extrapolated volition"?What is "evidential decision theory"?What is "friendly AI"?What is "functional decision theory"?What is "greater-than-human intelligence"?What is "hedonium"?What is "logical decision theory"?What is "metaphilosophy" and how does it relate to AI safety?What is "narrow AI"?What is "superintelligence"?What is "transformative AI"?What is "whole brain emulation"?What is AI Safety via Debate?What is Aligned AI / Stuart Armstrong working on?What is Anthropic's approach to LLM alignment?What is Artificial General Intelligence and what will it look like?What is Conjecture's Scalable LLM Interpretability research adgenda?What is Conjecture's epistemology research agenda?What is Conjecture, and what is their team working on?What is David Krueger working on?What is Dylan Hadfield-Menell's thesis on?What is GPT-3?What is Goodhart's law?What is John Wentworth's plan?What is MIRI’s mission?What is Stampy's copyright?What is a "pivotal act"?What is a "quantilizer"?What is a "value handshake"?What is a canonical question on Stampy's Wiki?What is a duplicate question on Stampy's Wiki?What is a follow-up question on Stampy's Wiki?What is a verified account on Stampy's Wiki?What is an "agent"?What is an "intelligence explosion"?What is an "s-risk"?What is artificial general intelligence safety / AI alignment?What is causal decision theory?What is interpretability and what approaches are there?What is meant by "AI takeoff"?What is neural network modularity?What is the "control problem"?What is the "long reflection"?What is the "orthogonality thesis"?What is the "universal prior"?What is the "windfall clause"?What is the Center for Human Compatible AI (CHAI)?What is the Center on Long-Term Risk (CLR) focused on?What is the Stampy project?What is the difference between inner and outer alignment?What is the general nature of the concern about AI alignment?What is the probability of extinction from misaligned superintelligence?What kind of a challenge is solving AI alignment?What kind of questions do we want on Stampy?What milestones are there between us and AGI?What plausibly happens five years before and after AGI?What research is being done to align modern deep learning systems?What safety problems are associated with whole brain emulation?What should I read to learn about decision theory?What should be marked as a "related" question on Stampy's Wiki?What should be marked as a canonical answer on Stampy's Wiki?What should the first AGI systems be aligned to do?What sources of information can Stampy use?What subjects should I study at university to prepare myself for alignment research?What technical problems are MIRI working on?What technological developments could speed up AI progress?What would a "warning shot" look like?What would a good future with AGI look like?What would a good solution to AI alignment look like?What would a world shortly before AGI look like?What would be physically possible and desirable to have in an AI-built utopia?What's especially worrisome about autonomous weapons?What's meant by calling an AI "agenty" or "agentlike"?What’s a good AI alignment elevator pitch?When should I stamp an answer?When will an intelligence explosion happen?When will transformative AI be created?Where can I find all the features of Stampy's Wiki?Where can I find mentorship and advice for becoming a researcher?Where can I find people to talk to about AI alignment?Where can I find questions to answer for Stampy?Where can I learn about AI alignment?Where can I learn about interpretability?Which country will AGI likely be created by, and does this matter?Which military applications of AI are likely to be developed?Which organizations are working on AI alignment?Which organizations are working on AI policy?Which university should I study at if I want to best prepare for working on AI alignment?Who created Stampy?Who is Nick Bostrom?Who is Stampy?Why can't we just make a "child AI" and raise it?Why can't we just turn the AI off if it starts to misbehave?Why can't we simply stop developing AI?Why can’t we just use Asimov’s Three Laws of Robotics?Why can’t we just use natural language instructions?Why can’t we just “put the AI in a box” so that it can’t influence the outside world?Why can’t we just…Why do some AI researchers not worry about alignment?Why do we expect that a superintelligence would closely approximate a utility maximizer?Why do you like stamps so much?Why does AI need goals in the first place? Can’t it be intelligent without any agenda?Why does AI takeoff speed matter?Why does there seem to have been an explosion of activity in AI in recent years?Why don't we just not build AGI if it's so dangerous?Why is AGI dangerous?Why is AGI safety a hard problem?Why is AI alignment a hard problem?Why is AI safety important?Why is safety important for smarter-than-human AI?Why is the future of AI suddenly in the news? What has changed?Why might a maximizing AI cause bad outcomes?Why might a superintelligent AI be dangerous?Why might an AI do something that we don’t want it to, if it’s really so intelligent?Why might contributing to Stampy be worth my time?Why might people try to build AGI rather than stronger and stronger narrow AIs?Why might we expect a fast takeoff?Why might we expect a moderate AI takeoff?Why might we expect a superintelligence to be hostile by default?Why should I worry about superintelligence?Why should we prepare for human-level AI technology now rather than decades down the line when it’s closer?Why think that AI can outperform humans?Why work on AI safety early?Why would great intelligence produce great power?Why would we only get one chance to align a superintelligence?Will AGI be agentic?Will AI learn to be independent from people or will it always ask for our orders?Will an aligned superintelligence care about animals other than humans?Will superintelligence make a large part of humanity unemployable?Will there be a discontinuity in AI capabilities? If so, at what stage?Will there be an AI-assisted "long reflection" and how might it look?Will we ever build a superintelligence?Won’t AI be just like us?Would "warning shots" make a difference and, if so, would they be helpful or harmful?Would AI alignment be hard with deep learning?Would an AI create or maintain suffering because some people want it?Would an aligned AI allow itself to be shut down?Would donating small amounts to AI safety organizations make any significant difference?Would it improve the safety of quantilizers to cut off the top few percent of the distribution?Wouldn't a superintelligence be smart enough not to make silly mistakes in its comprehension of our instructions?Wouldn't a superintelligence be smart enough to know right from wrong?Wouldn't it be a good thing for humanity to die out?Wouldn't it be safer to only build narrow AIs?
Question:
Duplicate of:An existing question is a duplicate of a new one if it is reasonable to expect whoever asked the new question to be satisfied if they received an answer to the existing question instead.	A lot of concern appears to focus on human-level or “superintelligent” AI. Is that a realistic prospect in the foreseeable future?AIs aren’t as smart as rats, let alone humans. Isn’t it far too early to be worrying about this kind of thing?Any AI will be a computer program. Why wouldn't it just do what it's programmed to do?Are AI researchers trying to make conscious AI?Are Google, OpenAI, etc. aware of the risk?Are any major politicians concerned about this?Are expert surveys on AI safety available?Are there any AI alignment projects which governments could usefully put a very large amount of resources into?Are there any plausibly workable proposals for regulating or banning dangerous AI research?Are there promising ways to make AI alignment researchers smarter?Are there risk analysis methods, which may help to make the risk more quantifiable or clear?Are there types of advanced AI that would be safer than others?Aren't robots the real problem? How can AI cause harm if it has no ability to directly manipulate the physical world?Aren’t there some pretty easy ways to eliminate these potential problems?At a high level, what is the challenge of alignment that we must meet to secure a good future?Can AI be creative?Can an AI really be smarter than humans?Can humans stay in control of the world if human- or superhuman-level AI is developed?Can people contribute to alignment by using proof assistants to generate formal proofs?Can we add "friendliness" to any artificial intelligence design?Can we constrain a goal-directed AI using specified rules?Can we ever be sure that an AI is aligned?Can we get AGI by scaling up architectures similar to current ones, or are we missing key insights?Can we program the superintelligence to maximize human pleasure or satisfaction of human desires?Can we teach a superintelligence a moral code with machine learning?Can we tell an AI just to figure out what we want and then do that?Can we test an AI to make sure that it’s not going to take over and do harmful things after it achieves superintelligence?Can you give an AI a goal which involves “minimally impacting the world”?Can you stop an advanced AI from upgrading itself?Can't we just tell an AI to do what we want?Can’t we just program the superintelligence not to harm us?Considering how hard it is to predict the future, why do we think we can say anything useful about AGI today?Could AI have basic emotions?Could I contribute by offering coaching to alignment researchers? If so, how would I go about this?Could an AGI have already been created and currently be affecting the world?Could divesting from AI companies without good safety culture be useful, or would this be likely to have a negligible impact?Could emulated minds do AI alignment research?Could we build provably beneficial AI systems?Could we get significant biological intelligence enhancements long before AGI?Could we program an AI to automatically shut down if it starts doing things we don’t want it to?Could we tell the AI to do what's morally right?Could weak AI systems help with alignment research?Do you need a PhD to work on AI Safety?Does it make sense to focus on scenarios where change is rapid and due to a single actor, or slower and dependent on getting agreements between several relevant actors?Does the importance of AI risk depend on caring about transhumanist utopias?Even if we are rationally convinced about the urgency of existential AI risk, it can be hard to feel that emotionally because the danger is so abstract. How can this gap be bridged?How can I be a more productive student/researcher?How can I collect questions for Stampy?How can I contact the Stampy team?How can I contribute in the area of community building?How can I contribute to Stampy?How can I convince others and present the arguments well?How can I get hired by an organization working on AI alignment?How can I join the Stampy dev team?How can I support alignment researchers to be more productive?How can we interpret what all the neurons mean?How close do AI experts think we are to creating superintelligence?How could an intelligence explosion be useful?How could general intelligence be programmed into a machine?How could poorly defined goals lead to such negative outcomes?How difficult should we expect alignment to be?How do I add content from LessWrong / Effective Altruism Forum tag-wikis to Stampy?How do I form my own views about AI safety?How do I format answers on Stampy?How do I know whether I'm a good fit for work on AI safety?How do I stay motivated and productive?How do I stay updated about AI progress?How do organizations do adversarial training and red teaming?How do the incentives in markets increase AI risk?How does AI taking things literally contribute to alignment being hard?How does MIRI communicate their view on alignment?How does the current global microchip supply chain work, and who has political power over it?How does the stamp eigenkarma system work?How doomed is humanity?How fast will AI takeoff be?How good is the world model of GPT-3?How hard is it for an AGI to develop powerful nanotechnology?How important is research closure and OPSEC for capabilities-synergistic ideas?How is "intelligence" defined?How is AGI different from current AI?How is Beth Barnes evaluating LM power seeking?How is OpenAI planning to solve the full alignment problem?How is metaethics relevant to AI alignment?How is the Alignment Research Center (ARC) trying to solve Eliciting Latent Knowledge (ELK)?How likely are AI organizations to respond appropriately to the risks of their creations?How likely is an "intelligence explosion"?How likely is it that AGI will first be developed by a large established organization, rather than a small startup, an academic group or a government?How likely is it that an AI would pretend to be a human to further its goals?How likely is it that governments will play a significant role? What role would be desirable, if any?How long will it be until superintelligent AI is created?How might "acausal trade" affect alignment?How might AGI interface with cybersecurity?How might AGI kill people?How might a real-world AI system that receives orders in natural language and does what you mean look?How might a superintelligence socially manipulate humans?How might a superintelligence technologically manipulate humans?How might an "intelligence explosion" be dangerous?How might an AI achieve a seemingly beneficial goal via inappropriate means?How might non-agentic GPT-style AI cause an "intelligence explosion" or otherwise contribute to existential risk?How might things go wrong with AI even without an agentic superintelligence?How might we get from Artificial General Intelligence to a Superintelligent system?How might we reduce the chance of an AI arms race?How might we reduce the diffusion of dangerous AI technology to insufficiently careful actors?How much can we learn about AI with interpretability tools?How much resources did the processes of biological evolution use to evolve intelligent creatures?How possible (and how desirable) is it to change which path humanity follows to get to AGI?How powerful will a mature superintelligence be?How quickly could an AI go from the first indications of problems to an unrecoverable disaster?How quickly would the AI capabilities ecosystem adopt promising new advances in AI alignment?How should I change my financial investments in response to the possibility of transformative AI?How should I decide which quality level to attribute to a proposed question?How should I personally prepare for when transformative AI arrives?How software- and/or hardware-bottlenecked are we on AGI?How successfully have institutions managed risks from novel technology in the past?How tractable is it to get governments to play a good role (rather than a bad role) and/or to get them to play a role at all (rather than no role)?How would I know if AGI were imminent?How would we align an AGI whose learning algorithms / cognition look like human brains?How would we know if an AI were suffering?How would you explain the theory of Infra-Bayesianism?I want to help out AI alignment without necessarily making major life changes. What are some simple things I can do to contribute?I want to work on AI alignment. How can I get funding?I'm interested in working on AI safety. What should I do?If AGI comes from a new paradigm, how likely is it to arise late in the paradigm when it is already deployed at scale, versus early on when only a few people are exploring the idea?If AI takes over the world how could it create and maintain the infrastructure that humans currently provide?If I only care about helping people alive today, does AI safety still matter?If an AI became conscious, how would we ever know?If we solve alignment, are we sure of a good future?In "aligning AI with human values", which humans' values are we talking about?In what ways are real-world machine learning systems different from expected utility maximizers?Is AI alignment possible?Is AI safety research racing against capability research? If so, how can safety research get ahead?Is expecting large returns from AI self-improvement just following an exponential trend line off a cliff?Is it already too late to work on AI alignment?Is it likely that hardware will allow an exponential takeoff?Is it possible to block an AI from doing certain things on the Internet?Is it possible to code into an AI to avoid all the ways a given task could go wrong, and would it be dangerous to try that?Is large-scale automated AI persuasion and propaganda a serious concern?Is merging with AI through brain-computer interfaces a potential solution to safety problems?Is the UN concerned about existential risk from AI?Is the focus on the existential threat of superintelligent AI diverting too much attention from more pressing debates about AI in surveillance and the battlefield, and its potential effects on the economy?Is the question of whether we're living in a simulation relevant to AI safety? If so, how?Is there a Chinese AI safety community? Are there safety researchers working at leading Chinese AI labs?Is there a danger in anthropomorphizing AI’s and trying to understand them in human terms?Is there something useful we can ask governments to do for AI alignment?Is this about AI systems becoming malevolent or conscious and turning on us?Isn't it hard to make a significant difference as a person who isn't going to be a world-class researcher?Isn't it too soon to be working on AGI safety?Isn't the real concern AI being misused by terrorists or other bad actors?Isn't the real concern AI-enabled totalitarianism?Isn't the real concern autonomous weapons?Isn't the real concern technological unemployment?Isn’t AI just a tool like any other? Won’t it just do what we tell it to?Isn’t it immoral to control and impose our values on AI?I’d like to get deeper into the AI alignment literature. Where should I look?Might an "intelligence explosion" never occur?Might an aligned superintelligence force people to "upload" themselves, so as to more efficiently use the matter of their bodies?Might an aligned superintelligence force people to have better lives and change more quickly than they want?Might an aligned superintelligence immediately kill everyone and then go on to create a "hedonium shockwave"?Might attempting to align AI cause a "near miss" which results in a much worse outcome?Might humanity create astronomical amounts of suffering when colonizing the universe after creating an aligned superintelligence?Might trying to build a hedonium-maximizing AI be easier and more likely to work than trying for eudaimonia?OK, I’m convinced. How can I help?Once we notice that a superintelligence given a specific task is trying to take over the world, can’t we turn it off, reprogram it or otherwise correct the problem?Should I engage in political or collective action like signing petitions or sending letters to politicians?Should we expect "warning shots" before an unrecoverable catastrophe?Superintelligence sounds like science fiction. Do people think about this in the real world?This all seems rather abstract. Isn't promoting love, wisdom, altruism or rationality more important?To what extent are there meaningfully different paths to AGI, versus just one path?We already have psychopaths who are "misaligned" with the rest of humanity, but somehow we deal with them. Can't we do something similar with AI?We’re going to merge with the machines so this will never be a problem, right?What about AI concerns other than misalignment?What about having a human supervisor who must approve all the AI's decisions before executing them?What actions can I take in under five minutes to contribute to the cause of AI safety?What alignment strategies are scalably safe and competitive?What approaches are AI alignment organizations working on?What are "coherence theorems" and what do they tell us about AI?What are "human values"?What are "scaling laws" and how are they relevant to safety?What are "selection theorems" and can they tell us anything useful about the likely shape of AGI systems?What are Encultured working on?What are OpenAI Codex and GitHub Copilot?What are Scott Garrabrant and Abram Demski working on?What are alternate phrasings for?What are brain-computer interfaces?What are language models?What are likely to be the first transformative applications of AI?What are mesa-optimizers?What are plausible candidates for "pivotal acts"?What are some AI alignment research agendas currently being pursued?What are some good books about AGI safety?What are some good podcasts about AI alignment?What are some good resources on AI alignment?What are some helpful AI policy ideas?What are some important examples of specialised terminology in AI alignment?What are some objections to the importance of AI alignment?What are some of the leading AI capabilities organizations?What are some of the most impressive recent advances in AI capabilities?What are some open research questions in AI alignment?What are some practice or entry-level problems for getting into alignment research?What are some problems in philosophy that are related to AI safety?What are some specific open tasks on Stampy?What are the "win conditions"/problems that need to be solved?What are the differences between AGI, transformative AI and superintelligence?What are the differences between “AI safety”, “AGI safety”, “AI alignment” and “AI existential safety”?What are the different possible AI takeoff speeds?What are the different versions of decision theory?What are the editorial protocols for Stampy questions and answers?What are the leading theories in moral philosophy and which of them might be technically the easiest to encode into an AI?What are the potential benefits of AI as it grows increasingly sophisticated?What are the style guidelines for writing for Stampy?What assets need to be protected by/from the AI? Are "human values" sufficient for it?What beneficial things would an aligned superintelligence be able to do?What can I do to contribute to AI safety?What can we expect the motivations of a superintelligent machine to be?What convinced people working on AI alignment that it was worth spending their time on this cause?What could a superintelligent AI do, and what would be physically impossible even for it?What does Elon Musk think about AI safety?What does Evan Hubinger think of Deception + Inner Alignment?What does MIRI think about technical alignment?What does Ought aim to do?What does a typical work day in the life of an AI safety researcher look like?What does alignment failure look like?What does generative visualization look like in reinforcement learning?What does the scheme Externalized Reasoning Oversight involve?What evidence do experts usually base their timeline predictions on?What external content would be useful to the Stampy project?What harm could a single superintelligence do when it took so many humans to build civilization?What if technological progress stagnates and we never achieve AGI?What if we put the AI in a box and have a second, more powerful, AI with the goal of preventing the first one from escaping?What is "Do What I Mean"?What is "HCH"?What is "agent foundations"?What is "biological cognitive enhancement"?What is "coherent extrapolated volition"?What is "evidential decision theory"?What is "friendly AI"?What is "functional decision theory"?What is "greater-than-human intelligence"?What is "hedonium"?What is "logical decision theory"?What is "metaphilosophy" and how does it relate to AI safety?What is "narrow AI"?What is "superintelligence"?What is "transformative AI"?What is "whole brain emulation"?What is AI Safety via Debate?What is Aligned AI / Stuart Armstrong working on?What is Anthropic's approach to LLM alignment?What is Artificial General Intelligence and what will it look like?What is Conjecture's Scalable LLM Interpretability research adgenda?What is Conjecture's epistemology research agenda?What is Conjecture, and what is their team working on?What is David Krueger working on?What is Dylan Hadfield-Menell's thesis on?What is GPT-3?What is Goodhart's law?What is John Wentworth's plan?What is MIRI’s mission?What is Stampy's copyright?What is a "pivotal act"?What is a "quantilizer"?What is a "value handshake"?What is a canonical question on Stampy's Wiki?What is a duplicate question on Stampy's Wiki?What is a follow-up question on Stampy's Wiki?What is a verified account on Stampy's Wiki?What is an "agent"?What is an "intelligence explosion"?What is an "s-risk"?What is artificial general intelligence safety / AI alignment?What is causal decision theory?What is interpretability and what approaches are there?What is meant by "AI takeoff"?What is neural network modularity?What is the "control problem"?What is the "long reflection"?What is the "orthogonality thesis"?What is the "universal prior"?What is the "windfall clause"?What is the Center for Human Compatible AI (CHAI)?What is the Center on Long-Term Risk (CLR) focused on?What is the Stampy project?What is the difference between inner and outer alignment?What is the general nature of the concern about AI alignment?What is the probability of extinction from misaligned superintelligence?What kind of a challenge is solving AI alignment?What kind of questions do we want on Stampy?What milestones are there between us and AGI?What plausibly happens five years before and after AGI?What research is being done to align modern deep learning systems?What safety problems are associated with whole brain emulation?What should I read to learn about decision theory?What should be marked as a "related" question on Stampy's Wiki?What should be marked as a canonical answer on Stampy's Wiki?What should the first AGI systems be aligned to do?What sources of information can Stampy use?What subjects should I study at university to prepare myself for alignment research?What technical problems are MIRI working on?What technological developments could speed up AI progress?What would a "warning shot" look like?What would a good future with AGI look like?What would a good solution to AI alignment look like?What would a world shortly before AGI look like?What would be physically possible and desirable to have in an AI-built utopia?What's especially worrisome about autonomous weapons?What's meant by calling an AI "agenty" or "agentlike"?What’s a good AI alignment elevator pitch?When should I stamp an answer?When will an intelligence explosion happen?When will transformative AI be created?Where can I find all the features of Stampy's Wiki?Where can I find mentorship and advice for becoming a researcher?Where can I find people to talk to about AI alignment?Where can I find questions to answer for Stampy?Where can I learn about AI alignment?Where can I learn about interpretability?Which country will AGI likely be created by, and does this matter?Which military applications of AI are likely to be developed?Which organizations are working on AI alignment?Which organizations are working on AI policy?Which university should I study at if I want to best prepare for working on AI alignment?Who created Stampy?Who is Nick Bostrom?Who is Stampy?Why can't we just make a "child AI" and raise it?Why can't we just turn the AI off if it starts to misbehave?Why can't we simply stop developing AI?Why can’t we just use Asimov’s Three Laws of Robotics?Why can’t we just use natural language instructions?Why can’t we just “put the AI in a box” so that it can’t influence the outside world?Why can’t we just…Why do some AI researchers not worry about alignment?Why do we expect that a superintelligence would closely approximate a utility maximizer?Why do you like stamps so much?Why does AI need goals in the first place? Can’t it be intelligent without any agenda?Why does AI takeoff speed matter?Why does there seem to have been an explosion of activity in AI in recent years?Why don't we just not build AGI if it's so dangerous?Why is AGI dangerous?Why is AGI safety a hard problem?Why is AI alignment a hard problem?Why is AI safety important?Why is safety important for smarter-than-human AI?Why is the future of AI suddenly in the news? What has changed?Why might a maximizing AI cause bad outcomes?Why might a superintelligent AI be dangerous?Why might an AI do something that we don’t want it to, if it’s really so intelligent?Why might contributing to Stampy be worth my time?Why might people try to build AGI rather than stronger and stronger narrow AIs?Why might we expect a fast takeoff?Why might we expect a moderate AI takeoff?Why might we expect a superintelligence to be hostile by default?Why should I worry about superintelligence?Why should we prepare for human-level AI technology now rather than decades down the line when it’s closer?Why think that AI can outperform humans?Why work on AI safety early?Why would great intelligence produce great power?Why would we only get one chance to align a superintelligence?Will AGI be agentic?Will AI learn to be independent from people or will it always ask for our orders?Will an aligned superintelligence care about animals other than humans?Will superintelligence make a large part of humanity unemployable?Will there be a discontinuity in AI capabilities? If so, at what stage?Will there be an AI-assisted "long reflection" and how might it look?Will we ever build a superintelligence?Won’t AI be just like us?Would "warning shots" make a difference and, if so, would they be helpful or harmful?Would AI alignment be hard with deep learning?Would an AI create or maintain suffering because some people want it?Would an aligned AI allow itself to be shut down?Would donating small amounts to AI safety organizations make any significant difference?Would it improve the safety of quantilizers to cut off the top few percent of the distribution?Wouldn't a superintelligence be smart enough not to make silly mistakes in its comprehension of our instructions?Wouldn't a superintelligence be smart enough to know right from wrong?Wouldn't it be a good thing for humanity to die out?Wouldn't it be safer to only build narrow AIs?
Canonical answer:	's Answer to How quickly would the AI capabilities ecosystem adopt promising new advances in AI alignment?Abram Demski's Answer to Maximizers and Satisficers on 2019-08-23T16:05:49 by Martin VerrisinAnswer to A lot of concern appears to focus on human-level or “superintelligent” AI. Is that a realistic prospect in the foreseeable future?Answer to AIs aren’t as smart as rats, let alone humans. Isn’t it far too early to be worrying about this kind of thing?Answer to Any AI will be a computer program. Why wouldn't it just do what it's programmed to do?Answer to Are Google, OpenAI, etc. aware of the risk?Answer to Are there types of advanced AI that would be safer than others?Answer to Aren't robots the real problem? How can AI cause harm if it has no ability to directly manipulate the physical world?Answer to Aren’t there some pretty easy ways to eliminate these potential problems?Answer to At a high level, what is the challenge of alignment that we must meet to secure a good future?Answer to Can an AI really be smarter than humans?Answer to Can people contribute to alignment by using proof assistants to generate formal proofs?Answer to Can we add "friendliness" to any artificial intelligence design?Answer to Can we constrain a goal-directed AI using specified rules?Answer to Can we get AGI by scaling up architectures similar to current ones, or are we missing key insights?Answer to Can we program the superintelligence to maximize human pleasure or satisfaction of human desires?Answer to Can we teach a superintelligence a moral code with machine learning?Answer to Can we tell an AI just to figure out what we want and then do that?Answer to Can we test an AI to make sure that it’s not going to take over and do harmful things after it achieves superintelligence?Answer to Can you give an AI a goal which involves “minimally impacting the world”?Answer to Can you stop an advanced AI from upgrading itself?Answer to Can't we just tell an AI to do what we want?Answer to Can’t we just program the superintelligence not to harm us?Answer to Could AI have basic emotions?Answer to Could we program an AI to automatically shut down if it starts doing things we don’t want it to?Answer to Couldn’t we keep the AI in a box and never give it the ability to manipulate the external world?Answer to How can I collect questions for Stampy?Answer to How can I contact the Stampy team?Answer to How can I contribute to Stampy?Answer to How can I join the Stampy dev team?Answer to How can we interpret what all the neurons mean?Answer to How close do AI experts think we are to creating superintelligence?Answer to How could an intelligence explosion be useful?Answer to How could general intelligence be programmed into a machine?Answer to How could poorly defined goals lead to such negative outcomes?Answer to How difficult should we expect alignment to be?Answer to How do I add content from LessWrong / Effective Altruism Forum tag-wikis to Stampy?Answer to How do I form my own views about AI safety?Answer to How do I format answers on Stampy?Answer to How does AI taking things literally contribute to alignment being hard?Answer to How does the stamp eigenkarma system work?Answer to How doomed is humanity?Answer to How fast will AI takeoff be?Answer to How is "intelligence" defined?Answer to How is OpenAI planning to solve the full alignment problem?Answer to How likely is an "intelligence explosion"?Answer to How likely is it that an AI would pretend to be a human to further its goals?Answer to How might AGI kill people?Answer to How might Shard Theory help with alignment?Answer to How might a superintelligence socially manipulate humans?Answer to How might a superintelligence technologically manipulate humans?Answer to How might an "intelligence explosion" be dangerous?Answer to How might an AI achieve a seemingly beneficial goal via inappropriate means?Answer to How might non-agentic GPT-style AI cause an "intelligence explosion" or otherwise contribute to existential risk?Answer to How might things go wrong with AI even without an agentic superintelligence?Answer to How might we get from Artificial General Intelligence to a Superintelligent system?Answer to How quickly could an AI go from the first indications of problems to an unrecoverable disaster?Answer to I want to help out AI alignment without necessarily making major life changes. What are some simple things I can do to contribute?Answer to I want to work on AI alignment. How can I get funding?Answer to I'm interested in working on AI safety. What should I do?Answer to If AI takes over the world how could it create and maintain the infrastructure that humans currently provide?Answer to If I only care about helping people alive today, does AI safety still matter?Answer to If we solve alignment, are we sure of a good future?Answer to Is AI alignment possible?Answer to Is expecting large returns from AI self-improvement just following an exponential trend line off a cliff?Answer to Is it possible to block an AI from doing certain things on the Internet?Answer to Is it possible to code into an AI to avoid all the ways a given task could go wrong, and would it be dangerous to try that?Answer to Is large-scale automated AI persuasion and propaganda a serious concern?Answer to Is the focus on the existential threat of superintelligent AI diverting too much attention from more pressing debates about AI in surveillance and the battlefield, and its potential effects on the economy?Answer to Is there a danger in anthropomorphizing AI’s and trying to understand them in human terms?Answer to Is this about AI systems becoming malevolent or conscious and turning on us? (Plex's Answer to Is this about AI systems becoming malevolent or conscious and turning on us?)Answer to Is this about AI systems becoming malevolent or conscious and turning on us? (Answer to Is this about AI systems becoming malevolent or conscious and turning on us?)Answer to Isn’t AI just a tool like any other? Won’t it just do what we tell it to?Answer to I’d like to get deeper into the AI alignment literature. Where should I look?Answer to Might an "intelligence explosion" never occur?Answer to OK, I’m convinced. How can I help?Answer to Once we notice that a superintelligence given a specific task is trying to take over the world, can’t we turn it off, reprogram it or otherwise correct the problem?Answer to Superintelligence sounds like science fiction. Do people think about this in the real world?Answer to We’re going to merge with the machines so this will never be a problem, right?Answer to What approaches are AI alignment organizations working on?Answer to What are "human values"?Answer to What are "scaling laws" and how are they relevant to safety?Answer to What are Scott Garrabrant and Abram Demski working on?Answer to What are alternate phrasings for?Answer to What are brain-computer interfaces?Answer to What are language models?Answer to What are mesa-optimizers?Answer to What are some AI alignment research agendas currently being pursued?Answer to What are some good books about AGI safety?Answer to What are some good podcasts about AI alignment?Answer to What are some good resources on AI alignment?Answer to What are some objections to the importance of AI alignment?Answer to What are some of the most impressive recent advances in AI capabilities?Answer to What are some specific open tasks on Stampy?Answer to What are the differences between “AI safety”, “AGI safety”, “AI alignment” and “AI existential safety”?Answer to What are the different possible AI takeoff speeds?Answer to What are the ethical challenges related to whole brain emulation?Answer to What are the main sources of AI existential risk?Answer to What are the potential benefits of AI as it grows increasingly sophisticated?Answer to What are the style guidelines for writing for Stampy?Answer to What can I do to contribute to AI safety?Answer to What can we expect the motivations of a superintelligent machine to be?Answer to What does Elon Musk think about AI safety?Answer to What exactly is AGI and what will it look like?Answer to What harm could a single superintelligence do when it took so many humans to build civilization?Answer to What is "HCH"?Answer to What is "biological cognitive enhancement"?Answer to What is "coherent extrapolated volition"?Answer to What is "evidential decision theory"?Answer to What is "friendly AI"?Answer to What is "functional decision theory"?Answer to What is "greater-than-human intelligence"?Answer to What is "hedonium"?Answer to What is "narrow AI"?Answer to What is "superintelligence"? (Scott's Answer to What is "superintelligence"?)Answer to What is "superintelligence"? (Answer to What is "superintelligence"?)Answer to What is "transformative AI"?Answer to What is "whole brain emulation"?Answer to What is AI Safety via Debate?Answer to What is AI safety?Answer to What is Anthropic's approach to LLM alignment?Answer to What is Artificial General Intelligence safety/alignment?Answer to What is Conjecture's Scalable LLM Interpretability research adgenda?Answer to What is Conjecture's epistemology research agenda?Answer to What is GPT-3?Answer to What is Goodhart's law?Answer to What is MIRI’s mission?Answer to What is Stampy's copyright?Answer to What is a "quantilizer"?Answer to What is a "value handshake"?Answer to What is a canonical question on Stampy's Wiki?Answer to What is a duplicate question on Stampy's Wiki?Answer to What is a follow-up question on Stampy's Wiki?Answer to What is an "agent"?Answer to What is an "intelligence explosion"?Answer to What is an "s-risk"?Answer to What is artificial general intelligence safety / AI alignment?Answer to What is causal decision theory?Answer to What is interpretability and what approaches are there?Answer to What is meant by "AI takeoff"?Answer to What is neural network modularity?Answer to What is the "control problem"?Answer to What is the "long reflection"?Answer to What is the "orthogonality thesis"?Answer to What is the "windfall clause"?Answer to What is the Center on Long-Term Risk (CLR) focused on?Answer to What is the DeepMind's safety team working on?Answer to What is the Stampy project?Answer to What is the general nature of the concern about AI alignment?Answer to What is the goal of Simulacra Theory?Answer to What kind of questions do we want on Stampy?Answer to What links are especially valuable to share on social media or other contexts?Answer to What safety problems are associated with whole brain emulation?Answer to What should I read to learn about decision theory?Answer to What should be marked as a canonical answer on Stampy's Wiki?Answer to What sources of information can Stampy use?Answer to What training programs and courses are available for AGI safety?Answer to What would a good future with AGI look like?Answer to What would a good solution to AI alignment look like?Answer to When should I stamp an answer?Answer to When will an intelligence explosion happen?Answer to When will transformative AI be created?Answer to Where can I find all the features of Stampy's Wiki?Answer to Where can I find people to talk to about AI alignment?Answer to Where can I find questions to answer for Stampy?Answer to Where can I learn about AI alignment?Answer to Where can I learn about interpretability?Answer to Who created Stampy?Answer to Who is Nick Bostrom?Answer to Who is Stampy?Answer to Why can't we just make a "child AI" and raise it?Answer to Why can't we just turn the AI off if it starts to misbehave?Answer to Why can't we simply stop developing AI?Answer to Why can’t we just use Asimov’s Three Laws of Robotics?Answer to Why can’t we just use natural language instructions?Answer to Why can’t we just… (Answer to Why can’t we just…)Answer to Why can’t we just… (Plex's Answer to Why can’t we just…)Answer to Why do we expect that a superintelligence would closely approximate a utility maximizer?Answer to Why does AI need goals in the first place? Can’t it be intelligent without any agenda?Answer to Why does AI takeoff speed matter?Answer to Why don't we just not build AGI if it's so dangerous?Answer to Why is AGI safety a hard problem?Answer to Why is AI alignment a hard problem?Answer to Why is AI safety important?Answer to Why is safety important for smarter-than-human AI?Answer to Why might a maximizing AI cause bad outcomes?Answer to Why might a superintelligent AI be dangerous?Answer to Why might contributing to Stampy be worth my time?Answer to Why might people try to build AGI rather than stronger and stronger narrow AIs?Answer to Why might we expect a fast takeoff?Answer to Why might we expect a moderate AI takeoff?Answer to Why might we expect a superintelligence to be hostile by default?Answer to Why not just put it in a box?Answer to Why should I worry about superintelligence?Answer to Why should we prepare for human-level AI technology now rather than decades down the line when it’s closer?Answer to Why think that AI can outperform humans?Answer to Why would great intelligence produce great power?Answer to Why would we only get one chance to align a superintelligence?Answer to Will an aligned superintelligence care about animals other than humans?Answer to Will we ever build a superintelligence?Answer to Would AI alignment be hard with deep learning?Answer to Would an aligned AI allow itself to be shut down?Answer to Would donating small amounts to AI safety organizations make any significant difference?Answer to Would it improve the safety of quantilizers to cut off the top few percent of the distribution?Answer to Wouldn't a superintelligence be smart enough not to make silly mistakes in its comprehension of our instructions?Answer to Wouldn't a superintelligence be smart enough to know right from wrong? (Answer to Wouldn't a superintelligence be smart enough to know right from wrong?)Answer to Wouldn't a superintelligence be smart enough to know right from wrong? (Aprillion's Answer to Wouldn't a superintelligence be smart enough to know right from wrong?)Answer to Wouldn't it be a good thing for humanity to die out?Answers wanting brief versionsAprillion's Answer to 10 Reasons to Ignore AI Safety on 2021-04-16T16:50:15 by cwjalexAprillion's Answer to 8Dbaybled8D's question on Intro to AI SafetyAprillion's Answer to Agustin Doige's question on Avoiding Positive Side EffectsAprillion's Answer to AkantorJojo's question on Intro to AI SafetyAprillion's Answer to Alliotte Raphael's question on Intro to AI SafetyAprillion's Answer to Can AI be creative?Aprillion's Answer to Daniel Buzovský's question on Where do we go nowAprillion's Answer to Dorda Giovex's question on Real Inner MisalignmentAprillion's Answer to Is it possible to limit an AGI from full access to the internet?Aprillion's Answer to Isn’t AI just a tool like any other?Aprillion's Answer to Jakub Mintal's question on Mesa-Optimizers 2Aprillion's Answer to Math Magician's question on Are AI Risks like Nuclear Risks?Aprillion's Answer to Mera Flynn's question on The Windfall ClauseAprillion's Answer to Mesa-Optimizers on 2021-02-17T11:05:43 by Lepus LunarisAprillion's Answer to Mesa-Optimizers on 2021-02-17T17:36:20 by Robert KAprillion's Answer to Mesa-Optimizers on 2021-02-18T14:51:23 by Alexander HarrisAprillion's Answer to Mesa-Optimizers on 2021-03-06T00:27:29 by Loz ShamlerAprillion's Answer to Might trying to build a hedonium-maximizing AI be easier and more likely to work than trying for eudaimonia?Aprillion's Answer to Riccardo manfrin's question on Mesa-OptimizersAprillion's Answer to Samuel Sandeen's question on Intro to AI SafetyAprillion's Answer to Smo1k's question on Mesa-Optimizers 2Aprillion's Answer to Smrt fašizmu's question on Maximizers and SatisficersAprillion's Answer to TestingAprillion's Answer to What's meant by calling an AI "agenty" or "agentlike"?Aprillion's Answer to Wouldn't it be safer to only build narrow AIs?Aprillion's Answer to hedgehog3180's question on The Windfall ClauseAtilla's Answer to When writing tools (as a programmer), how do I know what is okay to share and what isn't?Augustus Caesar's Answer to George Michael Sherry's question on Pascal's MuggingAugustus Caesar's Answer to Instrumental Convergence on 2021-02-24T05:56:14 by WILL DAugustus Caesar's Answer to James Tenney's question on Intro to AI SafetyAugustus Caesar's Answer to Mesa-Optimizers on 2021-02-19T21:47:02 by milpAugustus Caesar's Answer to Mesa-Optimizers on 2021-02-23T14:17:01 by androkguzAugustus Caesar's Answer to Mesa-Optimizers on 2021-02-24T01:55:01 by frozenbagel16Augustus Caesar's Answer to Mesa-Optimizers on 2021-02-24T11:12:04 by somenameAugustus Caesar's Answer to Mesa-Optimizers on 2021-04-12T11:45:03 by Fanny10000Augustus Caesar's Answer to Quantilizers on 2020-12-13T22:43:03 by TheWhiteWolfAugustus Caesar's Answer to Quantilizers on 2020-12-14T14:52:51 by fiziwigAugustus Caesar's Answer to Quantilizers on 2020-12-14T18:31:12 by Progressor 4wardAugustus Caesar's Answer to Quantilizers on 2020-12-30T05:52:51 by MarkAugustus Caesar's Answer to René's question on Reward ModelingAugustus Caesar's Answer to Unknown User's question on Intro to AI SafetyBeamnode's Answer to Does the importance of AI risk depend on caring about transhumanist utopias?CarlFeynman's Answer to Dismythed & JWA's question on The Orthogonality ThesisCasejp's Answer to Should I engage in political or collective action like signing petitions or sending letters to politicians?Casejp's Answer to What if we put the AI in a box and have a second, more powerful, AI with the goal of preventing the first one from escaping?ChaosAlpha's Answer to Toby Buckley's question on Mesa-OptimizersChlorokin's Answer to Could emulated minds do AI alignment research?Chlorokin's Answer to What are "coherence theorems" and what do they tell us about AI?Chlorokin's Answer to What if we put the AI in a box and have a second, more powerful, AI with the goal of preventing the first one from escaping?Chlorokin's Answer to What is "Do What I Mean"?Chlorokin's Answer to What is a "pivotal act"?Chlorokin's Answer to Will superintelligence make a large part of humanity unemployable?Command Master's Answer to M A's question on Real Inner MisalignmentCommand Master's Answer to Seeker.87's question on Real Inner MisalignmentCyberByte's Answer to How long will it be until superintelligent AI is created?Damaged's Answer to Bootleg Jones's question on Intro to AI SafetyDamaged's Answer to Ceelvain's question on Intro to AI SafetyDamaged's Answer to Geoffry Gifari's question on Steven Pinker on AIDamaged's Answer to Henry Goodman's question on Real Inner MisalignmentDamaged's Answer to Luka Rapava's question on The Orthogonality ThesisDamaged's Answer to Maccollo's question on Video Title UnknownDamaged's Answer to Matbmp's question on Intro to AI SafetyDamaged's Answer to Milan Mašát's question on Video Title UnknownDamaged's Answer to Rob Stringer's question on Mesa-OptimizersDamaged's Answer to Ryan Paton's question on Intro to AI SafetyDamaged's Answer to Sophrosynicle's question on Real Inner MisalignmentDamaged's Answer to Tim Peterson's question on AI learns to Create Cat PicturesDamaged's Answer to What can AGI do? on 2021-03-04T08:30:55 by Luka RapavaDamaged's Answer to Уэстерн Спай's question on Are AI Risks like Nuclear Risks?Dpaleka's Answer to Are there any courses on technical AI safety topics?Dropinthesea's Answer to What actions can I take in under five minutes to contribute to the cause of AI safety?Dropinthesea's Answer to What is "Do What I Mean"?Evhub's Answer to Mesa-Optimizers on 2021-02-18T00:06:17 by poketopa1234FLI's FAQFilip's Answer to Are AI researchers trying to make conscious AI?Filip's Answer to Do you need a PhD to work on AI Safety?Filip's Answer to In "aligning AI with human values", which humans' values are we talking about?Filip's Answer to Isn't it too soon to be working on AGI safety?Filip's Answer to We already have psychopaths who are "misaligned" with the rest of humanity, but somehow we deal with them. Can't we do something similar with AI?Filip's Answer to What about having a human supervisor who must approve all the AI's decisions before executing them?Filip's Answer to What are the differences between AGI, transformative AI and superintelligence?Filip's Answer to Will AI learn to be independent from people or will it always ask for our orders?Gelisam's Answer to ( ͡° ͜ʖ ͡°)'s question on Real Inner MisalignmentGelisam's Answer to Nomentir Alque Nomintio's question on Mesa-OptimizersGelisam's Answer to Quantilizers on 2020-12-13T22:20:31 by octavio echeverriaGelisam's Answer to Quantilizers on 2020-12-14T05:37:51 by Joshua HillerupGelisam's Answer to Quantilizers on 2020-12-14T10:02:10 by Julia HenriquesGelisam's Answer to Quantilizers on 2020-12-14T10:54:03 by Kolop315Gelisam's Answer to Quantilizers on 2020-12-21T07:14:00 by SerenaculaGelisam's Answer to Quantilizers on 2020-12-21T16:09:27 by MrLeoniuGelisam's Answer to Quantilizers on 2021-01-02T12:24:40 by jonseahGelisam's Answer to Quantilizers on 2021-02-09T18:00:54 by Jon BrayGelisam's Answer to Quantilizers on 2021-04-17T22:22:17.016139 by UnknownJJ Hep's Answer to Quantilizers on 2020-12-24T19:36:39 by Luke MillsJamespetts's Answer to Maximizers and Satisficers on 2021-02-20T17:58:35 by Donald EngelmannJeremyg's Answer to What milestones are there between us and AGI?Linnea's Answer to What are OpenAI Codex and GitHub Copilot?Linnea's Answer to What are the ethical challenges related to whole brain emulation?Luke Muehlhauser's Answer to What is "superintelligence"?MIRI's Answer to How long will it be until superintelligent AI is created?MIRI's Answer to Why can’t we just “put the AI in a box” so that it can’t influence the outside world?MIRI's Answer to Why work on AI safety early?Magdalena's Answer to What does generative visualization look like in reinforcement learning?Magdalena's Answer to What is artificial general intelligence safety / AI alignment?Magdalena's Answer to What is the difference between inner and outer alignment?Magdalena's Answer to What subjects should I study at university to prepare myself for alignment research?Matthew1970's Answer to What are the editorial protocols for Stampy questions and answers?Morpheus's Answer to Is it already too late to work on AI alignment?Morpheus's Answer to MattettaM's question on Maximizers and SatisficersMurphant's Answer to Could I contribute by offering coaching to alignment researchers? If so, how would I go about this?Murphant's Answer to Could we tell the AI to do what's morally right?Murphant's Answer to Do AIs suffer?Murphant's Answer to How can I contribute in the area of community building?Murphant's Answer to How likely is it that governments will play a significant role? What role would be desirable, if any?Murphant's Answer to How much resources did the processes of biological evolution use to evolve intelligent creatures?Murphant's Answer to Might an aligned superintelligence force people to have better lives and change more quickly than they want?Murphant's Answer to What are plausible candidates for "pivotal acts"?Murphant's Answer to What are some important examples of specialised terminology in AI alignment?Murphant's Answer to What are the "win conditions"/problems that need to be solved?Murphant's Answer to What is "metaphilosophy" and how does it relate to AI safety?Murphant's Answer to What's especially worrisome about autonomous weapons?Nattfrosten's Answer to What are language models?Nattfrosten's Answer to What are mesa-optimizers?Nico Hill2's Answer to Will we ever build a superintelligence?NotaSentientAI's Answer to Why not just put it in a box?Plex's Answer to 10 Reasons to Ignore AI Safety on 2021-03-08T13:55:51 by james scPlex's Answer to AkantorJojo's question on Intro to AI SafetyPlex's Answer to Aren't robots the real problem? How can AI cause harm if it has no ability to directly manipulate the physical world?Plex's Answer to Gedelijan's question on Real Inner MisalignmentPlex's Answer to Heisenmountain B's question on The Orthogonality ThesisPlex's Answer to How can I convince others and present the arguments well?Plex's Answer to How does the field of AI Safety want to accomplish its goal of preventing existential risk?Plex's Answer to How long will it be until superintelligent AI is created?Plex's Answer to How long will it be until transformative AI is created?Plex's Answer to I'm not convinced AI would be a severe threat to humanity. Why are you so sure?Plex's Answer to Instrumental Convergence on 2019-09-07T18:05:31 by Tyler GustPlex's Answer to Jay Ayerson's question on Intro to AI SafetyPlex's Answer to Killer Robot Arms Race on 2021-02-20T14:59:31 by ChironPlex's Answer to Marc Bollinger's question on Mesa-OptimizersPlex's Answer to Mesa-Optimizers on 2021-02-21T11:55:30 by Piñata OblongataPlex's Answer to Mesa-Optimizers on 2021-02-22T04:26:43 by Jorel FerminPlex's Answer to Mesa-Optimizers on 2021-02-22T11:35:00 by Damien AsmodeusPlex's Answer to Mesa-Optimizers on 2021-02-23T07:23:02 by ChrysippusPlex's Answer to Mesa-Optimizers on 2021-02-23T17:49:04 by Will HolmesPlex's Answer to Mesa-Optimizers on 2021-02-23T19:00:23 by aforcemorepowerfulPlex's Answer to Mesa-Optimizers on 2021-02-24T04:50:30 by Jonathon ChambersPlex's Answer to Mesa-Optimizers on 2021-02-24T19:05:36 by Solomon UckoPlex's Answer to Mesa-Optimizers on 2021-03-01T22:38:48 by Iagoba ApellanizPlex's Answer to Mesa-Optimizers on 2021-03-02T04:09:47 by Harsh DeshpandePlex's Answer to Mesa-Optimizers on 2021-03-11T09:34:01 by HTIDtrickyPlex's Answer to Nachis04's question on Intro to AI SafetyPlex's Answer to Niels Peppelaar's question on 10 Reasons to Ignore AI SafetyPlex's Answer to Quantilizers on 2020-12-13T20:52:21 by owen heckmannPlex's Answer to Quantilizers on 2020-12-13T21:45:39 by DragonSheepPlex's Answer to Quantilizers on 2020-12-13T21:53:33 by Bastiaan CnossenPlex's Answer to Quantilizers on 2020-12-13T21:53:49 by cmilkauPlex's Answer to Quantilizers on 2020-12-13T21:55:08 by loligesgamePlex's Answer to Quantilizers on 2020-12-13T21:58:44 by Vincent GrangePlex's Answer to Quantilizers on 2020-12-13T22:29:50 by Qwerty and AzertyPlex's Answer to Quantilizers on 2020-12-13T23:11:49 by DragonSheepPlex's Answer to Quantilizers on 2020-12-13T23:59:24 by NixiturPlex's Answer to Quantilizers on 2020-12-14T02:46:17 by M KellyPlex's Answer to Quantilizers on 2020-12-14T03:52:16 by James BarclayPlex's Answer to Quantilizers on 2020-12-14T05:12:00 by Jeremy HoffmanPlex's Answer to Quantilizers on 2020-12-14T06:05:58 by Paulo Van HuffelPlex's Answer to Quantilizers on 2020-12-14T07:32:09 by Taras PylypenkoPlex's Answer to Quantilizers on 2020-12-14T22:59:02 by PanzerkampfwagenPlex's Answer to Quantilizers on 2020-12-15T01:44:10 by AdibasWakfuPlex's Answer to Quantilizers on 2020-12-15T12:08:55 by Life HappensPlex's Answer to Quantilizers on 2020-12-15T23:08:13 byPlex's Answer to Quantilizers on 2020-12-19T13:13:33 by SocialDownclimberPlex's Answer to Quantilizers on 2020-12-19T13:37:50 by Alex WebbPlex's Answer to Quantilizers on 2020-12-19T18:22:43 by YezpahrPlex's Answer to Quantilizers on 2020-12-25T18:03:37 by Timothy HansenPlex's Answer to Quantilizers on 2020-12-30T17:26:14 by Sean PedersenPlex's Answer to Quantilizers on 2021-01-11T09:32:08 by Underrated1Plex's Answer to Quantilizers on 2021-02-18T15:39:07 by Spoon Of DoomPlex's Answer to Quantilizers on 2021-02-19T12:19:50 by Shantanu OjhaPlex's Answer to Quantilizers on 2021-02-20T12:55:45 by Marcus AntoniusPlex's Answer to Quantilizers on 2021-02-22T21:54:31 by James PettsPlex's Answer to Ranibow Sprimkle's question on Instrumental ConvergencePlex's Answer to Reitze Jansen's question on Mesa-OptimizersPlex's Answer to Safe Exploration on 2020-12-06T21:48:38 by Rares RotarPlex's Answer to Samuel Hvidager's question on Intro to AI SafetyPlex's Answer to TackerTacker's question on Mesa-Optimizers 2Plex's Answer to Testfollowup2Plex's Answer to TestingPlex's Answer to Testing rename second?Plex's Answer to The Orthogonality Thesis on 2019-04-20T04:57:23 by EncipheredPlex's Answer to The Windfall Clause on 2020-07-07T07:10:36 by boarattackboarPlex's Answer to The Windfall Clause on 2020-12-13T14:49:52 by the DecoyPlex's Answer to Tommy karrick's question on Mesa-OptimizersPlex's Answer to Use of Utility Functions on 2017-04-27T20:50:28 by William DyePlex's Answer to Use of Utility Functions on 2021-02-26T11:44:06 by Michael MoranPlex's Answer to What is "agent foundations"?Plex's Answer to What is a verified account on Stampy's Wiki?Plex's Answer to What is everyone working on in AI alignment?Plex's Answer to What’s a good AI alignment elevator pitch?Plex's Answer to Will there be a discontinuity in AI capabilities? If so, at what stage?Plex's Answer to jhjkhgjhfgjg jgjyfhdhbfjhg's question on Mesa-Optimizers 2Plex's second Answer to Test's question?Plex2's Answer to Testing rename second?QZ's Answer to Where can I find mentorship and advice for becoming a researcher?QueenDaisy's Answer to Are any major politicians concerned about this?QueenDaisy's Answer to Might an aligned superintelligence force people to "upload" themselves, so as to more efficiently use the matter of their bodies?QueenDaisy's Answer to What could a superintelligent AI do, and what would be physically impossible even for it?Quintin Pope's Answer to Will superintelligence make a large part of humanity unemployable?Redshift's Answer to In "aligning AI with human values", which humans' values are we talking about?Robert hildebrandt's Answer to Quantilizers on 2020-12-14T01:27:48 by MoleoRobert hildebrandt's Answer to Quantilizers on 2020-12-14T03:13:46 by SlimThrullRobert hildebrandt's Answer to Quantilizers on 2020-12-14T03:47:06 by snigwithaswordRobert hildebrandt's Answer to Quantilizers on 2020-12-14T04:04:30 by Noah McCannRobert hildebrandt's Answer to Quantilizers on 2020-12-14T11:23:40 by illesizsRobert hildebrandt's Answer to Quantilizers on 2020-12-16T06:22:32 by ChrysippusRobert.hildebrandt's Answer to Mesa-Optimizers on 2021-02-18T00:28:56 by LoliShocksRobert.hildebrandt's Answer to Mesa-Optimizers on 2021-02-18T05:37:03 by Irun SRobert.hildebrandt's Answer to Mesa-Optimizers on 2021-02-18T13:35:33 by Peter SmytheRobert.hildebrandt's Answer to WNJ: Think of AGI like a Corporation? on 2021-02-21T09:03:12 by ChedimRobertskmiles's Answer to A Commenter's question on What can AGI do?Robertskmiles's Answer to AI Safety Gridworlds 2 on 2020-06-02T00:45:31 by Wylliam JuddRobertskmiles's Answer to Alan W's question on Intro to AI SafetyRobertskmiles's Answer to Alessandrə Rustichelli's question on Intro to AI SafetyRobertskmiles's Answer to Are expert surveys on AI safety available?Robertskmiles's Answer to Avoiding Negative Side Effects on 2020-11-17T01:48:43 by Neological GamerRobertskmiles's Answer to Channel Introduction on 2021-04-07T23:33:04 by Robert MilesRobertskmiles's Answer to Instrumental Convergence on 2020-05-18T23:25:44 by phil guerRobertskmiles's Answer to Is merging with AI through brain-computer interfaces a potential solution to safety problems?Robertskmiles's Answer to Killer Robot Arms Race on 2020-06-06T11:20:21 by DaVince21Robertskmiles's Answer to Maor Eitan's question on Intro to AI SafetyRobertskmiles's Answer to Maximizers and Satisficers on 2019-09-01T08:11:48 by Paper BenniRobertskmiles's Answer to Mesa-Optimizers on 2021-02-19T08:59:09 by valbermRobertskmiles's Answer to Mesa-Optimizers on 2021-02-25T10:23:18 by RaukGorthRobertskmiles's Answer to NINJA NAJM's question on Avoiding Negative Side EffectsRobertskmiles's Answer to Quantilizers on 2020-12-13T22:12:28 by Markus JohanssonRobertskmiles's Answer to Quantilizers on 2020-12-14T18:12:54 by Blah BlahRobertskmiles's Answer to Quantilizers on 2020-12-15T18:18:01 by mrsuperguy2073Robertskmiles's Answer to Quantilizers on 2020-12-25T23:07:12 by Peter FranzRobertskmiles's Answer to Quantilizers on 2021-02-19T12:19:50 by Shantanu OjhaRobertskmiles's Answer to Quantilizers on 2021-02-24T23:54:57 by Nathan KouvalisRobertskmiles's Answer to Rob Sokolowski's question on AI Safety Gridworlds 2Robertskmiles's Answer to Steven Pinker on AI on 2020-08-23T02:40:09 by Xystem 4Robertskmiles's Answer to Superintelligence Mod for Civilization V on 2019-04-11T22:16:10 by Mateja PetrovicRobertskmiles's Answer to The Orthogonality Thesis on 2019-04-18T19:04:56 by Jan BamRobertskmiles's Answer to The Orthogonality Thesis on 2020-10-13T22:10:08 by Juan Pablo Garibotti AriasRobertskmiles's Answer to The Orthogonality Thesis on 2021-02-21T04:49:23 by peterbrehmjRobertskmiles's Answer to Uiytt's question on Intro to AI SafetyRobertskmiles's Answer to Use of Utility Functions on 2020-09-23T08:51:14 by Amaar QuadriRobertskmiles's Answer to What can AGI do? on 2020-03-30T09:18:52 by Jade GortonRobertskmiles's Answer to What can AGI do? on 2020-12-19T06:19:12 by FiraroRobertskmiles's Answer to Where do we go now on 2020-05-12T20:06:14 by Musthegreat 94Robertskmiles's Answer to ZT1ST's question on Instrumental ConvergenceRobertskmiles's Answer to Ian's question on Maximizers and SatisficersRoseMcClelland's Answer to How do you figure out model performance scales?RoseMcClelland's Answer to How does MIRI communicate their view on alignment?RoseMcClelland's Answer to How is Beth Barnes evaluating LM power seeking?RoseMcClelland's Answer to How is the Alignment Research Center (ARC) trying to solve Eliciting Latent Knowledge (ELK)?RoseMcClelland's Answer to How would we align an AGI whose learning algorithms / cognition look like human brains?RoseMcClelland's Answer to How would you explain the theory of Infra-Bayesianism?RoseMcClelland's Answer to What are Encultured working on?RoseMcClelland's Answer to What does Evan Hubinger think of Deception + Inner Alignment?RoseMcClelland's Answer to What does MIRI think about technical alignment?RoseMcClelland's Answer to What does Ought aim to do?RoseMcClelland's Answer to What does the scheme Externalized Reasoning Oversight involve?RoseMcClelland's Answer to What is Aligned AI / Stuart Armstrong working on?RoseMcClelland's Answer to What is Conjecture, and what is their team working on?RoseMcClelland's Answer to What is David Krueger working on?RoseMcClelland's Answer to What is Dylan Hadfield-Menell's thesis on?RoseMcClelland's Answer to What is FAR's theory of change?RoseMcClelland's Answer to What is Future of Humanity Instititute working on?RoseMcClelland's Answer to What is John Wentworth's plan?RoseMcClelland's Answer to What is Refine?RoseMcClelland's Answer to What is Truthful AI's approach to improve society?RoseMcClelland's Answer to What is an adversarial oversight scheme?RoseMcClelland's Answer to What is the Center for Human Compatible AI (CHAI)?RoseMcClelland's Answer to What is the purpose of the Visible Thoughts Project?RoseMcClelland's Answer to What language models are Anthropic working on?RoseMcClelland's Answer to What other organizations are working on technical AI alignment?RoseMcClelland's Answer to What projects are CAIS working on?RoseMcClelland's Answer to What projects are Redwood Research working on?RoseMcClelland's Answer to What work is Redwood doing on LLM interpretability?RoseMcClelland's Answer to Who is Jacob Steinhardt and what is he working on?RoseMcClelland's Answer to Who is Sam Bowman and what is he working on?Self-modification and wireheadingSeverin's Answer to How can I be a more productive student/researcher?Severin's Answer to Isn't the real concern AI being misused by terrorists or other bad actors?Severin's Answer to What are the leading theories in moral philosophy and which of them might be technically the easiest to encode into an AI?SlimeBunnyBat's Answer to 5astelija's question on Mesa-Optimizers 2SlimeBunnyBat's Answer to Andy Gee's question on Mesa-Optimizers 2SlimeBunnyBat's Answer to Ansatz66's question on Intro to AI SafetySlimeBunnyBat's Answer to Arthur Wittmann's question on Killer Robot Arms RaceSlimeBunnyBat's Answer to Ben Crulis's question on Mesa-OptimizersSlimeBunnyBat's Answer to Etienne Maheu's question on Intro to AI SafetySlimeBunnyBat's Answer to IkarusKK's question on Real Inner MisalignmentSlimeBunnyBat's Answer to Isn't the real concern technological unemployment?SlimeBunnyBat's Answer to Marcelo Pinheiro's question on Real Inner MisalignmentSlimeBunnyBat's Answer to Marie Rentergent's question on Mesa-Optimizers 2SlimeBunnyBat's Answer to Mesa-Optimizers on 2021-02-17T08:00:03 by James CrewdsonSlimeBunnyBat's Answer to Mesa-Optimizers on 2021-02-17T10:26:59 by KoroSlimeBunnyBat's Answer to Midhunraj R's question on QuantilizersSlimeBunnyBat's Answer to Nerd Herd's question on Real Inner MisalignmentSlimeBunnyBat's Answer to Oliver Bergau's question on QuantilizersSlimeBunnyBat's Answer to Robert Tuttle's question on Mesa-Optimizers 2SlimeBunnyBat's Answer to Sigmata0's question on Intro to AI SafetySlimeBunnyBat's Answer to SlimeBunnyBat's Answer to Drekpaprika's question on Intro to AI SafetySlimeBunnyBat's Answer to Son of a Beech's question on The Orthogonality ThesisSlimeBunnyBat's Answer to Stellar Lake System's question on The Orthogonality ThesisSocial Christancing's Answer to G T's question on Mesa-OptimizersSocial Christancing's Answer to Mesa-Optimizers on 2021-02-17T12:57:20 by X3 KJSocial Christancing's Answer to Mesa-Optimizers on 2021-02-23T01:55:12 by Sebastian GramszSocial Christancing's Answer to Mesa-Optimizers on 2021-02-28T20:30:04 by DodoDojoSocial Christancing's Answer to Quantilizers on 2021-02-22T13:59:07 by Marshall WhiteSocial Christancing's Answer to Siranut usawasutsakorn's question on Mesa-Optimizers 2Social Christancing's Answer to Socially unacceptable's question on The Orthogonality ThesisStargate9000's Answer to Mesa-Optimizers on 2021-02-28T14:11:32 by andybaldmanStargate9000's Answer to Mesa-Optimizers on 2021-03-13T23:08:08 by Tomasz RogalaStargate9000's Answer to Quantilizers on 2021-02-18T15:39:07 by Spoon Of DoomStargate9000's Answer to Quantilizers on 2021-03-09T18:02:04 by Blackmage89Stargate9000's Answer to The Orthogonality Thesis on 2021-02-27T23:46:44 by Stellar Lake SystemSudonym's Answer to Famitory's question on Intro to AI SafetySudonym's Answer to Instrumental Convergence on 2020-06-09T18:06:22 by YuvalSudonym's Answer to Iterated Distillation and Amplification on 2021-01-07T19:03:28 by Keenan PepperSudonym's Answer to Mesa-Optimizers on 2021-03-09T20:02:31 by CormanSudonym's Answer to Mesa-Optimizers on 2021-03-09T21:29:31 by MetsuryuSudonym's Answer to Quantilizers on 2020-12-13T22:53:59 by J MSudonym's Answer to Quantilizers on 2020-12-14T01:34:53 by boobshartSudonym's Answer to Quantilizers on 2020-12-15T05:08:10 by DarkestMirroredSudonym's Answer to Quantilizers on 2020-12-15T09:14:48 by Samuel WoodsSudonym's Answer to Quantilizers on 2020-12-15T16:27:49 by NutwitSudonym's Answer to Quantilizers on 2020-12-16T05:57:09 by Adrian RegenfußSudonym's Answer to Quantilizers on 2020-12-18T00:46:12 by Wilco VerhoefSudonym's Answer to Quantilizers on 2020-12-25T04:37:20 by Daniel MKSudonym's Answer to Quantilizers on 2020-12-25T11:41:42 by PianoShowSudonym's Answer to Quantilizers on 2020-12-26T16:22:16 by SongbirdSudonym's Answer to Quantilizers on 2021-01-02T15:40:56 by wertyuiopSudonym's Answer to Quantilizers on 2021-01-05T19:27:35 by kade99TVSudonym's Answer to Quantilizers on 2021-01-09T16:33:43 by StephenSudonym's Answer to Reward Hacking Reloaded on 2020-10-26T01:46:26 by Julian DanzerSudonym's Answer to Steven Pinker on AI on 2020-05-13T19:07:46 by kilroy1964Sudonym's Answer to The Orthogonality Thesis on 2019-04-15T17:07:34 by echoesSudonym's Answer to The Orthogonality Thesis on 2020-12-27T23:05:56 by Miguel BorromeoSudonym's Answer to Uberchops's question on QuantilizersSudonym's Answer to WNJ: Think of AGI like a Corporation? on 2020-06-05T00:00:24 by Clayton VogesSudonym's Answer to What does alignment failure look like?TJ6K's Answer to What beneficial things would an aligned superintelligence be able to do?TapuZuko's Answer to Is the question of whether we're living in a simulation relevant to AI safety? If so, how?TapuZuko's Answer to Isn't the real concern autonomous weapons?TapuZuko's Answer to Might an aligned superintelligence immediately kill everyone and then go on to create a "hedonium shockwave"?Tinytitan's Answer to Could we get significant biological intelligence enhancements long before AGI?U8k's Answer to onje berdy's question on Real Inner MisalignmentYaakov's Answer to What are the different versions of decision theory?Yaakov's Answer to Which organizations are working on AI alignment?Yevgeniy Andreyevich's Answer to Lapis Salamander's question on Intro to AI SafetyYevgeniy Andreyevich's Answer to Rich Traube's question on WNJ: Think of AGI like a Corporation?Yevgeniy Andreyevich's Answer to afla light's question on 10 Reasons to Ignore AI SafetyYevgeniy's Answer to Ted Archer's question on Maximizers and Satisficers
Asked on Discord?Has this been posted to a Discord channel for the community to try answering?		Canonical question?Is this an idealized version of a common question, rather than an exact example of a question someone asked?		Follow-up question?Is this question only relevant as a followup, so should not appear in search?		For Rob?Is this just for Rob (e.g. someone asking Rob about an aspect of his video production)		Not a question?Is this not a question? (e.g. someone trying to chat with Stampy, or just has a URL with a ? in it)		Out of Scope?Is this outside of Stampy's scope (i.e. not directly relevant to AI existential safety or the Stampy project)?
Asker:
Date:When was this question asked?	JanuaryFebruaryMarchAprilMayJuneJulyAugustSeptemberOctoberNovemberDecember
Origin:Where is this question from?
Asked on video:	"Don't Fear The Terminator" - Yann LeCun on Facebook10 Reasons to Ignore AI Safety9 Examples of Specification GamingA Response to Steven Pinker on AIAI Safety GridworldsAI Safety at EAGlobal2017 ConferenceAI That Doesn't Try Too Hard - Maximizers and SatisficersAI learns to Create ̵K̵Z̵F̵ ̵V̵i̵d̵e̵o̵s̵ Cat Pictures: Papers in Two MinutesAre AI Risks like Nuclear Risks?Avoiding Negative Side Effects: Concrete Problems in AI Safety part 1Avoiding Positive Side Effects: Concrete Problems in AI Safety part 1.5Channel IntroductionDeceptive Misaligned Mesa-Optimisers? It's More Likely Than You Think...Empowerment: Concrete Problems in AI Safety part 2Experts' Predictions about the Future of AIFriend or Foe? AI Safety Gridworlds extra bitHow to Keep Improving When You're Better Than Any Teacher - Iterated Distillation and AmplificationIntelligence and Stupidity: The Orthogonality ThesisIntro to AI Safety, RemasteredIs AI Safety a Pascal's Mugging?MAXIMUM OVERGEORGIAMuch Better Stampy Test VideoPC Build Video!Predicting AI: RIP Prof. Hubert DreyfusQuantilizers: AI That Doesn't Try Too HardReading and Commenting On Pinker's ArticleRespectabilityReward Hacking Reloaded: Concrete Problems in AI Safety Part 3.5Reward Hacking: Concrete Problems in AI Safety Part 3Safe Exploration: Concrete Problems in AI Safety Part 6Scalable Supervision: Concrete Problems in AI Safety Part 5Sharing the Benefits of AI: The Windfall ClauseStatus ReportSuperintelligence Mod for Civilization VThe OTHER AI Alignment Problem: Mesa-Optimizers and Inner AlignmentThe other "Killer Robot Arms Race" Elon Musk should worry aboutTraining AI Without Writing A Reward Function, with Reward ModellingUntitled2We Were Right! Real Inner MisalignmentWhat Can We Do About Reward Hacking?: Concrete Problems in AI Safety Part 4What can AGI do? I/O and SpeedWhat's the Use of Utility Functions?Where do we go now?Why Not Just: Raise AI Like Kids?Why Not Just: Think of AGI Like a Corporation?Why Would AI Want to do Bad Things? Instrumental Convergence

[Show Advanced Options]

[Hide Advanced Options]

---

Edit summary

This is a minor edit Notify me of changes

Cancel