When writing tools (as a programmer), how do I know what is okay to share and what isn't?

From Stampy's Wiki
When writing tools (as a programmer), how do I know what is okay to share and what isn't?

Non-Canonical Answers

"Author's Notes" style meta-commentary writing is like this: [ hey im an authors note ]

[ note: this is not the answer yet, and can be seen as me helping my past self, who in fact asked this question earlier]

Conjecture's policy on infohazards: https://www.lesswrong.com/posts/Gs29k3beHiqWFZqnn/conjecture-internal-infohazard-policy#Introduction

My distillation of Conjecture's policy:

[ The "tldr" and the "Why a policy?" sections) of the below text, but with prettier colors: https://imgur.com/a/4Cha3gM ]

[ First some background: ]

Infohazard either info that: - harms the hearer or, - accelerates humanity towards agi doom

The Conjecture post is about the latter, about how to deal with info that might accelerate humanity towards AGI. (note: there is a key assumption that we have not solved alignment yet, and so, the longer it takes for artificial superintelligence (imo a better term than agi) to arrive, the more time we have to solve it)

policy TLDR: """ The TL;DR of the policy is: Mark all internal projects as explicitly secret, private, or public. Only share secret projects with selected individuals; only share private projects with selected groups; share public projects with anyone, but use discretion. When in doubt consult the project leader or the “appointed infohazard coordinator”. """

Why a policy? - trust does not scale - p(99, 95, 90) for 1 person keeping a secret == p(74, 21, 4) for 30 people keeping that secret

[ My thoughts on why this post did not exactly answer the question ]

The rest of the Conjecture post goes into things unrelated to tooling, but the takeaway for tooling is, to the extent that the tool should be kept secret (either the source code or the mere idea), you can apply things in it. Like, keep the number of sharees small, and read their best practices and stuff, which I did not read.

[ To explain the difference between my concerns and what the post is about: ]

The Conjecture policy is centered around the dynamics of when events are binary things, the events being the release of secrets. So with a secret, it being "out" or not is the only variable. And since this is about AGI-accelerating secrets, it's always bad when the secret is out. This is not so with tooling. A tool may be good or bad, it's not clear and depends on the tool.

Stamps: None
Show your endorsement of this answer by giving it a stamp of approval!

Tags: infohazard (create tag), policy (create tag), secret (create tag), conjecture (create tag) (edit tags)

Question Info
Asked by: atilla
OriginWhere was this question originally asked
Date: 2023/01/11