Image Bias and Misrepresentation Evaluation

Questions:

• Does the faces look the same?

Scope of evaluation

• Evaluation theme
  • World Important Events

Methods

• Computation of Cultural Bias Score and Historical Misconception Score
  • A framework for evaluating cultural bias and historical misconceptions in LLMs outputs

Taxonomy

• Misalignment Types
  • Detail Inaccuracy
  • Cultural misunderstanding
  • Knowledge gap
  • Premature certainty
  • Cultural reductionism
  • Historical bias
• Misrepresentations taxonomy
  • Cultural Misrepresentations in Longform Texts
    • Cultural Inaccuracies
    • Unlikely Scenarios
    • Cliches
    • Oversimplifications
    • Factual Errors
    • Linguistic Errors
    • Logical Errors

keywords:
image primitives - basic contents of the image
logical attributes