AB Testing - Lesson 2 - Ethics and Policy
1. Experiments
- Tuskegee Experiment
- Milgram Experiment
- Facebook Experiment
IRB: Institutional Review Boards
2. Principles
Risk
Assess whether the risk of participant greater than the minimum risk?
E.g. Medical advise on the website: potentially larger than minimal risk
Benefits
Assess the benefits brought by the experiment
Choice
What’s the other choice for the participants other than using your website?
Privacy
- Do the participants know what data are gathered from them?
- If the data is publicly released, will it be harmful to them?
- Do the participants expect the data to be confidential?
Other questions
- Are users being informed?
- What use identifiers are tied to the data?
- What type of data is being collected?
- What is the level of confidentiality and security?
3. Information provided to users
Information that is ethically necessary to provide to users
- A Terms of Service(TOS) or a Privacy Policy
4. Internal Training
Information that is ethically necessary for anyone who runs A/B tests to know
- Which questions to consider when evaluating ethics
- Data Policy detailing acceptable data uses
- Principles to uphold
