Stampy
The Stampy project is open effort to build a comprehensive FAQ about artificial intelligence existential safety—the field trying to make sure that when we build superintelligent artificial systems they are aligned with human values so that they do things compatible with our survival and flourishing.
We're also building a cleaner web UI for readers and a bot interface.
The goals of the project are to:
- Offer a one-stop-shop for high-quality answers to common questions about AI alignment.
- Let people answer questions in a way which scales, freeing up researcher time while allowing more people to learn from a reliable source.
- Make external resources more easy to find by having links to them connected to a search engine which gets smarter the more it's used.
- Provide a form of legitimate peripheral participation for the AI Safety community, as an on-boarding path with a flexible level of commitment.
- Encourage people to think, read, and talk about AI alignment while answering questions, creating a community of co-learners who can give each other feedback and social reinforcement.
- Provide a way for budding researchers to prove their understanding of the topic and ability to produce good work.
- Collect data about the kinds of questions people actually ask and how they respond, so we can better focus resources on answering them.
- Track reactions on messages so we can learn which answers need work.
- Identify missing external content to create.
If you would like to help out, join us on the Discord and either jump right into editing or read get involved for answers to common questions.
| Main Question: What is the Stampy project? (edit question) (edit answer) |
Canonically answered
How do I format answers on Stampy?
Items on lists start with *, numbered lists with #
- For external links use [ followed directly by the URL, a space, then display text and finally a ] symbol
- e.g. [https://www.example.com External link text] gives External link text
- For internal links write the page title wrapped in [[]]s
- e.g. [[What is the Stampy project?]] gives What is the Stampy project?. Including a pipe symbol followed by display text e.g. [[What is the Stampy project?┊Display Text]] allows you to show different Display Text.
- (ref)Reference notes go inside these tags(/ref)[1]
- If you post the raw URL of an image from imgur it will be displayed.[2] You can reduce file compression if you get an account. Note that you need the image itself, right click -> copy image address to get it
- To embed a YouTube video, use (youtube)APsK8NST4qE(/youtube) with the video ID of the target video.
- Start with ** or ## for double indentation
- Three 's around text - Bold
- Two 's around text Italic - Italic
How can I join the Stampy dev team?
The development team works on multiple projects in support of Stampy. Currently, these projects include:
- Stampy UI, which is made mostly in TypeScript.
- The Stampy Wiki, which is made mostly in PHP and JavaScript.
- The Stampy Bot, which is made in Python.
However, even if you don’t specialize in any of these areas, do reach out if you would like to help.
To join, please contact our Project Manager, plex. You can reach him on discord at plex#1874. He will be able to point your skills in the right direction to help in the most effective way possible.
How does the stamp eigenkarma system work?
If someone posts something good - something that shows insight, knowledge of AI Safety, etc. - give the message or answer a stamp of approval! Stampy keeps track of these, and uses them to decide how much he likes each user. You can ask Stampy (in a PM if you like), "How many stamps am I worth?", and he'll tell you.
If something is really very good, especially if it took a lot of work/effort, give it a gold stamp. These are worth 5 regular stamps!
Note that stamps aren't just 'likes', so please don't give stamps to say "me too" or "that's funny" etc. They're meant to represent knowledge, understanding, good judgement, and contributing to the discord. You can use 💯 or ✔️ for things you agree with, 😂 or 🤣 for funny things etc.
Your stamp points determine how much say you have if there are disagreements on Stampy content, which channels you have permission to post to, your voting power for approving YouTube replies, and whether you get to invite people.
Notes on stamps and stamp points
- Stamps awarded by people with a lot of stamp points are worth more
- Awarding people stamps does not reduce your stamp points
- New users who have 0 stamp points can still award stamps, they just have no effect. But it's still worth doing because if you get stamp points later, all your previous votes are retroactively updated!
- Yes, this was kind of tricky to implement! Stampy actually stores how many stamps each user has awarded to every other user, and uses that to build a system of linear scalar equations which is then solved with numpy.
- Each user has stamp points, and also gives a score to every other user they give stamps to the scores sum to 1 so if I give user A a stamp, my score for them will be 1.0, if I then give user B a stamp, my score for A is 0.5 and B is 0.5, if I give another to B, my score for A goes to 0.3333 and B to 0.66666 and so on
- Score is "what proportion of the stamps I've given have gone to this user"
- Everyone's stamp points is the sum of (every other user's score for them, times that user's stamp points) so the way to get points is to get stamps from people who have points
- Rob is the root of the tree, he got one point from Stampy
- So the idea is the stamp power kind of flows through the network, giving people points for posting things that I thought were good, or posting things that "people who posted things I thought were good" thought were good, and so on ad infinitum so for posting YouTube comments, Stampy won't send the comment until it has enough stamps of approval. Which could be a small number of high-points users or a larger number of lower-points users
- Stamps given to yourself or to stampy do nothing
So yeah everyone ends up with a number that basically represents what Stampy thinks of them, and you can ask him "how many stamps am I worth?" to get that number
so if you have people a, b, and c, the points are calculated by:
a_points = (bs_score_for_a * b_points) + (cs_score_for_a * c_points)
b_points = (as_score_for_b * a_points) + (cs_score_for_b * c_points)
c_points = (as_score_for_c * a_points) + (bs_score_for_c * b_points)
which is tough because you need to know everyone else's score before you can calculate your own
but actually the system will have a fixed point - there'll be a certain arrangement of values such that every node has as much flowing out as flowing in - a stable configuration
so you can rearrange
(bs_score_for_a * b_points) + (cs_score_for_a * c_points) - a_points = 0
(as_score_for_b * a_points) + (cs_score_for_b * c_points) - b_points = 0
(as_score_for_c * a_points) + (bs_score_for_c * b_points) - c_points = 0
or, for neatness:
( -1 * a_points) + (bs_score_for_a * b_points) + (cs_score_for_a * c_points) = 0
(as_score_for_b * a_points) + ( -1 * b_points) + (cs_score_for_b * c_points) = 0
(as_score_for_c * a_points) + (bs_score_for_c * b_points) + ( -1 * c_points) = 0
and this is just a system of linear scalar equations that you can throw at numpy.linalg.solve
(you add one more equation that says rob_points = 1, so there's some place to start from)
there should be one possible distribution of points such that all of the equations hold at the same time, and numpy finds that by linear algebra magic beyond my very limited understanding
but as far as I can tell you can have all the cycles you want!
(I actually have the scores sum to slightly less than 1, to have the stamp power slightly fade out as it propagates, just to make sure it doesn't explode. But I don't think I actually need to do that)
and yes this means that any time anyone gives a stamp to anyone, ~everyone's points will change slightly
And yes this means I'm recalculating the matrix and re-solving it for every new stamp, but computers are fast and I'm sure there are cheaper approximations I could switch to later if necessary
What kind of questions do we want on Stampy?
Stampy is focused specifically on AI existential safety (both introductory and technical questions), but does not aim to cover general AI questions or other topics which don't interact strongly with the effects of AI on humanity's long-term future. More technical questions are also in our scope, though replying to all possible proposals is not feasible and this is not a place to submit detailed ideas for evaluation.
We are interested in:
- Introductory questions closely related to the field e.g.
- "How long will it be until transformative AI arrives?"
- "Why might advanced AI harm humans?"
- Technical questions related to the field e.g.
- "What is Cooperative Inverse Reinforcement Learning?"
- "What is Logical Induction useful for?"
- Questions about how to contribute to the field e.g.
- "Should I get a PhD?"
- "Where can I find relevant job opportunities?"
More good examples can be found at canonical questions.
We do not aim to cover:
- Aspects of AI Safety or fairness which are not strongly relevant to existential safety e.g.
- "How should self-driving cars weigh up moral dilemmas"
- "How can we minimize the risk of privacy problems caused by machine learning algorithms?"
- Extremely specific and detailed questions the answering of which is unlikely to be of value to more than a single person e.g.
- "What if we did <multiple paragraphs of dense text>? Would that result in safe AI?"
We will generally not delete out-of-scope content, but it will be reviewed as low priority to answer, not be marked as a canonical question, and not be served to readers by on Stampy's UI.
What sources of information can Stampy use?
As well as pulling human written answers to AI alignment questions from Stampy's Wiki, Stampy can:
- Search for AI safety papers e.g. "stampy, what's that paper about corrigibility?"
- Search for videos e.g. "what's that video where Rob talks about mesa optimizers, stampy?"
- Calculate with Wolfram Alpha e.g. "s, what's the square root of 345?"
- Search DuckDuckGo and return snippets
- And (at least in the patron Discord) falls back to polling GPT-3 to answer uncaught questions
We're also building a cleaner web UI for readers and a bot interface.
What should be marked as a canonical answer on Stampy's Wiki?
Canonical answers may be served to readers by Stampy, so only answers which have a reasonably high stamp score should be marked as canonical. All canonical answers are open to be collaboratively edited and updated, and they should represent a consensus response (written from the Stampy Point Of View) to a question which is within Stampy's scope.
Answers to questions from YouTube comments should not be marked as canonical, and will generally remain as they were when originally written since they have details which are specific to an idiosyncratic question. YouTube answers may be forked into wiki answers, in order to better respond to a particular question, in which case the YouTube question should have its canonical version field set to the new more widely useful question.
How can I contribute to Stampy?
If you're not already there, join the Discord where the contributors hang out.
The main ways you can help are to answer questions or add questions, or help to review questions, review answers, or improve answers (instructions for helping out with each of these tasks are on the linked pages). You could also join the dev team if you have programming skills.
How can I collect questions for Stampy?
As well as simply adding your own questions over at ask question, you could also message your friends with something like:
Hi,
I'm working on a project to create a comprehensive FAQ about AI alignment (you can read about it here https://stampy.ai/wiki/Stampy%27s_Wiki if interested). We're looking for questions and I thought you may have some good ones. If you'd be willing to write up a google doc with you top 5-10ish questions we'd be happy to write a personalized FAQ for you. https://stampy.ai/wiki/Scope explains the kinds of questions we're looking for.
Thanks!
and maybe bring the google doc to a Stampy editing session so we can collaborate on answering them or improving your answers to them.
When should I stamp an answer?
You show stamp an answer when you think it is accurate and well presented enough that you'd be happy to see it served to readers by Stampy.
Stampy is a character invented by Robert Miles and developed by the Stampy dev team. He is a stamp collecting robot, a play on clippy from the the paperclip maximizer thought experiment.
Stampy is designed to teach people about the risks of unaligned artificial intelligence, and facilitate a community of co-learners who build his FAQ database.
Where can I find all the features of Stampy's Wiki?
What are some good resources on AI alignment?
- Rob's YouTube videos (Computerphile appearances)
- AI Safety Papers database - Search and interface for the TAI Safety Bibliography
- AGI Safety Fundamentals Course
- Alignment Forum tags
- The Alignment Newsletter (and database sheet)
- Chapters of Bostrom's Superintelligence online (Initial paper which Superintelligence grew from)
- AI Alignment pages on Arbital
- Much more on AI Safety Support (feel free to integrate useful things from there to here)
- Vika's resources list
- AI safety technical courses, reading lists, and curriculums
- AI Safety Intro blog
- Stampy's canonical answers list
- All content produced on this wiki is released under the CC-BY-SA 4.0 license. Exceptions for unattributed use may be granted by admins, contact plex for inquiries.
- Questions from YouTube or other sources are reproduced with the intent of fair use, as derivative and educational material.
- Source code of https://ui.stampy.ai/ is released under MIT license
- Logo and visual design copyright is owned by Rob Miles, all rights reserved.
What are some specific open tasks on Stampy?
Other than the usual fare of writing and processing and organizing questions and answers, here are some specific open tasks:
- Porting over some of Steve Byrnes's FAQ on alignment
- Porting over content from Vael Gates's post
- Porting over QA pairs from https://www.lesswrong.com/posts/8c8AZq5hgifmnHKSN/agi-safety-faq-all-dumb-questions-allowed-thread
- Porting over some of https://aisafety.wordpress.com/
- Making sure we cover all of https://forum.effectivealtruism.org/posts/8JazqnCNrkJtK2Bx4/why-eas-are-skeptical-about-ai-safety#Recursive_self_improvement_seems_implausible and the responses
How do I add content from LessWrong / Effective Altruism Forum tag-wikis to Stampy?
You can include a live-updating version of many definitions from LW using the syntax on Template:TagDesc in the Answer field and Template:TagDescBrief on the Brief Answer field. Similarly, calling Template:TagDescEAF and Template:TagDescEAFBrief will pull from the EAF tag wiki.
When available this should be used as it reduces the duplication of effort and directs all editors to improving a single high quality source.
Why might contributing to Stampy be worth my time?
If you're looking for a shovel ready and genuinely useful task to further AI alignment without necessarily committing a large amount of time or needing deep specialist knowledge, we think Stampy is a great option!
Creating a high-quality single point of access where people can be onboarded and find resources around the alignment ecosystem seems likely to be high-impact. So, what makes us the best option?
- Unlike all other entry points to learning about alignment, we doge the trade-off between comprehensiveness and being overwhelmingly long with interactivity (tab explosion in one page!) and semantic search. Single document FAQs can't do this, so we built a system which can.
- We have the ability to point large numbers of viewers towards Stampy once we have the content, thanks to Rob Miles and his 100k+ subscribers, so this won't remain an unnoticed curiosity.
- Unlike most other entry points, we are open for volunteers to help improve the content.
- The main notable one which does is the LessWrong tag wiki, which hosts descriptions of core concepts. We strongly believe in not needlessly duplicating effort, so we're pulling live content from that for the descriptions on our own tag pages, and directing the edit links on those to the edit page on the LessWrong wiki.
You might also consider improving Wikipedia's alignment coverage or the LessWrong wiki, but we think Stampy has the most low-hanging fruit right now. Additionally, contributing to Stampy means being part of a community of co-learners who provide mentorship and encouragement to join the effort to give humanity a bight future. If you're an established researcher or have high-value things to do elsewhere in the ecosystem it might not be optimal to put much time into Stampy, but if you're looking for a way to get more involved it might well be.
What are alternate phrasings for?
Alternate phrasings are used to improve the semantic search which Stampy uses to serve people questions, by giving alternate ways to say a question which might trigger a match when the main wording won't. They should generally only be used when there is a significantly different wording, rather than for only very minor changes.
Dev team
Name |
Vision talk |
Github |
Trello |
Active? |
Notes / bio |
|---|---|---|---|---|---|
Aprillion |
video |
Aprillion |
yes |
yes |
experienced dev (Python, JS, CSS, ...) |
Augustus Caesar |
yes |
AugustusCeasar |
yes |
soon! |
Has some Discord bot experience |
Benjamin Herman |
no |
no (not needed) |
no |
no |
Helping with wiki design/css stuff |
ccstan99 |
no |
ccstan99 |
yes |
yes |
UI/UX designer |
chriscanal |
yes |
chriscanal |
yes |
yes |
experienced python dev |
Damaged |
no (not needed) |
no (not needed) |
no (not needed) |
yes |
experienced Discord bot dev, but busy with other projects. Can answer questions. |
plex |
yes |
plexish |
yes |
yes |
MediaWiki, plans, and coordinating people guy |
robertskmiles |
yes |
robertskmiles |
yes |
yes |
you've probably heard of him |
Roland |
yes |
levitation |
yes |
yes |
working on Semantic Search |
sct202 |
yes |
no (add when wiki is on github) |
yes |
yes |
PHP dev, helping with wiki extensions |
Social Christancing |
yes |
chrisrimmer |
yes |
maybe |
experienced linux sysadmin |
sudonym |
yes |
jmccuen |
yes |
yes |
systems architect, has set up a lot of things |
tayler6000 |
yes |
tayler6000 |
no |
yes |
Python and PHP dev, PenTester, works on Discord bot |
Editors
How can I contact the Stampy team?
The Rob Miles AI Discord is the hub of all things Stampy. If you want to be part of the project and don't have access yet, ask plex#1874 on Discord (or plex on wiki).
You can also talk to us on the public Discord! Try #suggestions or #general, depending on what you want to talk about.
Where can I find questions to answer for Stampy?
Answer questions collects all the questions we definitely want answers to, browse there and see if you know how to answer any of them.
What is a duplicate question on Stampy's Wiki?
An existing question is a duplicate of a new one if it is reasonable to expect whoever asked the new question to be satisfied if they received an answer to the existing question instead.
What are the style guidelines for writing for Stampy?
Avoid directly responding to the question in the answer, repeat the relevant part of the question instead. For example, if the question is "Can we do X", answer "We might be able to do X, if we can do Y", not "Yes, if we can manage Y". This way, the answer will also work for the questions "Why can't we do X" and "What would happen if we tried to do X".
Linking to external sites is strongly encouraged, one of the most valuable things Stampy can do is help people find other parts of the alignment information ecosystem.
Consider enclosing newly introduced terms, likely to be unfamiliar to many readers, in speech marks. If unsure, Google the term (in speech marks!) and see if it shows up anywhere other than LessWrong, the Alignment Forum, etc. Be judicious, as it's easy to use too many, but used carefully they can psychologically cushion newbies from a lot of unfamiliar terminology - in this context they're saying something like "we get that we're hitting you with a lot of new vocab, and you might not know what this term means yet".
When selecting related questions, there shouldn't be more than four unless there's a really good reason for that (some questions are asking for it, like the "Why can't we just..." question). It's also recommended to include at least one more "enticing" question to draw users in (relating to the more sensational, sci-fi, philosophical/ethical side of things) alongside more bland/neutral questions.
What is a canonical question on Stampy's Wiki?
Canonical questions are the questions which we've checked are in scope and not duplicates, so we want answers to them. They may be edited to represent a class of question more broadly, rather than keeping all their idosyncracies. Once they're answered canonically Stampy will serve them to readers.
What is a follow-up question on Stampy's Wiki?
Follow-up questions are responses to an answer which reader might have, either because they want more information or are providing information to Stampy about what they're looking for. We don't expect to have great coverage of the former for a long time because there will be so many, but hopefully we'll be able to handle some of the most common ones.
Non-canonical answers
What is a verified account on Stampy's Wiki?
Verified accounts are given to people who have clearly demonstrated understanding of AI Safety outside of this project, such as by being employed and vouched for by a major AI Safety organization or by producing high-impact research. Verified accounts may freely mark answers as canonical or not, regardless of how many Stamps the person has, to determine whether those answers are used by Stampy.
William Dye's question on Use of Utility Functions id:UgiF4JfGGvlE93gCoAEC
I just found this comment via the new Stampy wiki: https://stampy.ai/wiki/Main_Page which is the interface we'll be using to construct an FAQ using questions on Rob's channel as a base. Good idea, though it took us a few years to get to it, and did it in a slightly different form.
Unanswered canonical questions
