stampy
| Main Question: What is the Stampy project? (edit question) (edit answer) |
Description
The Stampy project is a volunteer effort to create a comprehensive FAQ on Artificial Intelligence existential safety, and a bot (stampy) capable of using the FAQ and other resources to educate people about AI alignment via an interactive natural language interface.
The goals of the project are to:
- Offer answers which are regularly improved and reviewed by our community
- Let people answer questions in a way which scales, freeing up the time of people who understand the field while allowing more people to learn from a reliable source
- Between the stamp eigenkarma system and giving verified researchers and other proven people power to promote or dis-promote answers, we'll try to reliably surface only answers which have been checked by someone who knows what they're talking about
- Make external resources more easy to find by encouraging lots of links out
- Provide a form of legitimate peripheral participation for the AI Safety community, as an on-boarding path for people who want to help
- Encourage people to think and read about AI alignment while trying to answer questions
- Create a community of co-learners who can give each other feedback and social reinforcement
- Collect data about the kinds of questions people actually ask and how they respond, so we can better focus resources on answering them
- Track reactions on messages so we can learn which answers need work
- Identify missing external content to create
Canonically answered
We are interested in:
- Introductory questions closely related to the field e.g.
- "How long will it be until transformative AI arrives?"
- "Why might advanced AI harm humans?"
- Technical questions related to the field e.g.
- "What is Cooperative Inverse Reinforcement Learning?"
- "What is Logical Induction useful for?"
- Questions about how to contribute to the field e.g.
- "Should I get a PhD?"
- "Where can I find relevant job opportunities?"
More good examples can be found at canonical questions.
We do not aim to cover:
- Aspects of AI Safety or fairness which are not strongly relevant to existential safety e.g.
- "How should self-driving cars weigh up moral dilemmas"
- "How can we minimize the risk of privacy problems caused by machine learning algorithms?"
- Extremely specific and detailed questions the answering of which is unlikely to be of value to more than a single person e.g.
- "What if we did <multiple paragraphs of dense text>? Would that result in safe AI?"
Verified accounts are given to people who have clearly demonstrated understanding of AI Safety outside of this project, such as by being employed and vouched for by a major AI Safety organization or by producing high-impact research. Verified accounts may freely mark answers as canonical or not, regardless of how many Stamps the person has, to determine whether those answers are used by Stampy.
As well as simply adding your own questions over at ask question, you could also message your friends with something like:
Hi,
I'm working on a project to create a comprehensive FAQ about AI alignment (you can read about it here https://stampy.ai/wiki/Stampy%27s_Wiki if interested). We're looking for questions and I thought you may have some good ones. If you'd be willing to write up a google doc with you top 5-10ish questions we'd be happy to write a personalized FAQ for you. https://stampy.ai/wiki/Scope explains the kinds of questions we're looking for.
Thanks!
and maybe bring the google doc to a Stampy editing session so we can collaborate on answering them or improving your answers to them.
If you're not already there, join the Discord where contributors hang out.
The main ways you can help are to answer questions or ask questions, or help to review answers or review questions. We're looking to cover everything in Stampy's scope. You could also join the dev team if you have programming skills.
One great thing you can do is write up a Google Doc with your top ~10 questions and post it to the Discord, or ask you friends to do the same (see follow-up question on collecting questions for a template message).
If you are a researcher or otherwise employed by an AI Safety focused organization, please contact us and we'll set you up with an account with extra privileges.
Here is a collection of helpful resources to link to while answering questions!
- Rob's YouTube videos (Computerphile appearances)
- AI Safety Papers database - Search and interface for the TAI Safety Bibliography
- Alignment Forum tags
- The Alignment Newsletter (and database sheet)
- Chapters of Bostrom's Superintelligence online
- AI Alignment pages on Arbital
- Much more on AI Safety Support (feel free to integrate useful things from there to here)
- Vika's resources list
- AGI Safety Fundamentals Course
- FHI's x-risk FAQ
- AI safety technical courses, reading lists, and curriculums
Imported FAQs (permission granted to use):
Canonical answers may be served to readers by Stampy, so only answers which have a reasonably high stamp score should be marked as canonical. All canonical answers are open to be collaboratively edited and updated, and they should represent a consensus response (written from the Stampy Point Of View) to a question which is within Stampy's scope.
Answers to YouTube questions should not be marked as canonical, and will generally remain as they were when originally written since they have details which are specific to an idiosyncratic question. YouTube answers may be forked into wiki answers, in order to better respond to a particular question, in which case the YouTube question should have its canonical version field set to the new more widely useful question.
The Stampy project is a volunteer effort to create a comprehensive FAQ on Artificial Intelligence existential safety, and a bot (stampy) capable of using the FAQ and other resources to educate people about AI alignment via an interactive natural language interface.
The goals of the project are to:
- Offer answers which are regularly improved and reviewed by our community
- Let people answer questions in a way which scales, freeing up the time of people who understand the field while allowing more people to learn from a reliable source
- Between the stamp eigenkarma system and giving verified researchers and other proven people power to promote or dis-promote answers, we'll try to reliably surface only answers which have been checked by someone who knows what they're talking about
- Make external resources more easy to find by encouraging lots of links out
- Provide a form of legitimate peripheral participation for the AI Safety community, as an on-boarding path for people who want to help
- Encourage people to think and read about AI alignment while trying to answer questions
- Create a community of co-learners who can give each other feedback and social reinforcement
- Collect data about the kinds of questions people actually ask and how they respond, so we can better focus resources on answering them
- Track reactions on messages so we can learn which answers need work
- Identify missing external content to create
An existing question is a duplicate of a new one if it is reasonable to expect whoever asked the new question to be satisfied if they received an answer to the existing question instead.
Canonical questions are the questions which we've checked are in scope and not duplicates, so we want answers to them. They may be edited to represent a class of question more broadly, rather than keeping all their idosyncracies. Once they're answered canonically Stampy will serve them to readers.
Follow-up questions are responses to an answer which reader might have, either because they want more information or are providing information to Stampy about what they're looking for. We don't expect to have great coverage of the former for a long time because there will be so many, but hopefully we'll be able to handle some of the most common ones.
Talk to us on Discord! Try #suggestions or #general, depending on what you want to talk about.
Alternate phrasings are used to improve the semantic search which Stampy uses to serve people questions, by giving alternate ways to say a question which might trigger a match when the main wording won't. They should generally only be used when there is a significantly different wording, rather than for only very minor changes.
Try to avoid directly referencing the wording of the question in the answer, in order to make the answer more robust to alternate phrasings. For example, that question might be "Can we do X" and the reply is "Yes, if we can manage Y", but then the question might be "why can't we do X" or "What would happen if we tried to do X" so the answer should be like "We might be able to do X, if we can do Y", which works for all of those.
Non-canonical answers
I just found this comment via the new Stampy wiki: https://stampy.ai/wiki/Main_Page which is the interface we'll be using to construct an FAQ using questions on Rob's channel as a base. Good idea, though it took us a few years to get to it, and did it in a slightly different form.
The Stamp Points System
If someone posts something good - something that shows insight, knowledge of AI Safety, etc. - give the message a stamp :stamp: of approval! @Stampy keeps track of these, and uses them to decide how much he likes each user. You can ask Stampy (in a PM if you like), "How many stamps am I worth?", and he'll tell you. If something is really very good, especially if it took a lot of work/effort, give it a gold stamp :goldstamp:. These are worth 5 regular stamps!
Note that stamps aren't just 'likes', so please don't give stamps to say "me too" or "that's funny" etc. They're meant to represent knowledge, understanding, good judgement, and contributing to the discord. You can use :100: or :heavy_check_mark: for things you agree with, :laughing: or :rofl: for funny things etc.
Your stamp points determine your voting power for approving YouTube replies, and in future probably other things, like getting invite links to share etc.
Notes on stamps and stamp points
- Stamps awarded by people with a lot of stamp points are worth more
- Awarding people stamps does not reduce your stamp points
- New users who have 0 stamp points can still award stamps, they just have no effect. But it's still worth doing because if you get stamp points later, all your previous votes are retroactively updated!
- Yes, this was kind of tricky to implement! Stampy actually stores how many stamps each user has awarded to every other user, and uses that to build a system of linear scalar equations which is then solved with numpy.
- When people post things that are insightful, that show good judgement or good knowledge of AI safety, you give them a :stamp:
- Each user has stamp points, and also gives a score to every other user they give stamps to the scores sum to 1 so if I give user A a stamp, my score for them will be 1.0, if I then give user B a stamp, my score for A is 0.5 and B is 0.5, if I give another to B, my score for A goes to 0.3333 and B to 0.66666 and so on
- Score is "what proportion of the stamps I've given have gone to this user"
- Everyone's stamp points is the sum of (every other user's score for them, times that user's stamp points) so the way to get points is to get stamps from people who have points
- Rob is the root of the tree, he got one point from stampy
- So the idea is the stamp power kind of flows through the network, giving people points for posting things that I thought were good, or posting things that "people who posted things I thought were good" thought were good, and so on ad infinitum so for posting youtube comments, stampy won't send the comment until it has enough stamps of approval. Which could be a small number of high-points users or a larger number of lower-points users
- Oh also, :goldstamp: is just equivalent to 5 stamps and stamps given to yourself or to stampy do nothing
So yeah everyone ends up with a number that basically represents what stampy thinks of them, and you can ask him "how many stamps am I worth?" to get that number
For technical details, see: https://discord.com/channels/677546901339504640/758062805810282526/781208566408413235
You show stamp an answer when you think it is accurate and well presented enough that you'd be happy to see it served to readers by Stampy.
Unanswered canonical questions
Unanswered non-canonical questions
