Dimension-by-dimension CID Rubric scores | Dim | Dimension | Score | Weight | Flag |
| D1 | Definitional Precision | 5 | 12% | 'Islamophobia' defined via example taxonomy without formal criteria |
| D2 | Classification Rigor | 5 | 18% | Reliability testing absent — report acknowledges this explicitly |
| D3 | Case Capture & Sampling | 4 | 15% | Closed-Universe Percentage Problem — no baseline, no symmetric monitoring |
| D4 | Coverage Symmetry | 5 | 15% | No symmetric monitoring of other mayoral candidates |
| D5 | Source Independence | 6 | 10% | — |
| D6 | Verification Standards | 5 | 18% | No raw data access — social media posts not archived with permalinks |
| D7 | Transparency & Governance | 4 | 5% | No funding disclosure — CSOH/IHL founder relationship not disclosed |
| D8 | Counter-Evidence | 6 | 7% | — |
| Composite Score | 5.3 | Deficient |
Metrics
- Denominator Rate
-
48%
22 of 46 numeric claims
Share of numeric claims that include a denominator or base rate. Low rates suggest missing context.
- Self-Citation Rate
-
19%
citations from org or affiliates
How often the report cites its own organization or close affiliates. High rates reduce source independence.
- Critical Flags
-
1
of 4 total flags
Flags at "high" or "severe" severity — methodological issues that materially affect the score.
Methodology Flags
Scope: The 450% increase and 72% extremist framing are percentages calculated within a pre-classified dataset of posts already identified as Islamophobic — not rates calculated against all political content about Mamdani. To put it plainly: if you start by collecting only posts you've already decided are hateful, a 450% increase means you found more hateful posts, not that 450% more of political discourse was hateful. Without a baseline count of all political content, these numbers cannot establish prevalence.
Scope: No funding disclosure in published report. CSOH/IHL shared founder relationship not disclosed, creating an undisclosed source independence consideration for D5.
Scope: Two coders classified 35,522 posts but no statistical test was run to measure how often they agreed. Without this, there is no way to know whether a different pair of coders would produce the same results. The report acknowledges this honestly.
Scope: Islamophobia classification taxonomy present but operational criteria for borderline cases not specified.
Scoring Notes
5/10 12% weight
'Islamophobia' defined via example taxonomy without formal criteria
Classification taxonomy is present but criteria for borderline cases are unspecified. Reproduces examples without systematic operationalization.
5/10 18% weight
Reliability testing absent — report acknowledges this explicitly
Two coders classified content but no agreement statistics were calculated to verify that they classified the same posts the same way. Partial credit awarded because the report acknowledges this gap rather than claiming reliability it did not test.
4/10 15% weight
Closed-Universe Percentage Problem — no baseline, no symmetric monitoring
35,522 posts represent the classified dataset, not total discourse. No monitoring of other candidates. No baseline of non-Islamophobic content.
5/10 15% weight
No symmetric monitoring of other mayoral candidates
Monitoring scope limited to Islamophobic content about one candidate. Cannot assess whether observed patterns are unusual for political campaigns.
6/10 10% weight
CSOH is the sole producer. CSOH/IHL shared founder not disclosed in this report.
5/10 18% weight
No raw data access — social media posts not archived with permalinks
Methodology does not document data preservation or access process. Replication not possible from published report.
4/10 5% weight
No funding disclosure — CSOH/IHL founder relationship not disclosed
Organization website does not list funders. IHL relationship not surfaced in this report despite relevance to source independence.
6/10 7% weight
Report acknowledges monitoring limitations at several points and explicitly flags that reliability testing was not conducted. Stronger on self-disclosure than most reports in this score range.
Citation Context
How this report's findings have been cited or applied after publication. Severity reflects the gap between what the report establishes and how it was represented.
Claimed scope: Documented prevalence and escalation of Islamophobia in the 2024 NYC mayoral race
Established scope: Internal composition statistics of pre-classified posts from one candidate's monitoring period
The 450% figure was reported in several outlets as a prevalence increase in political Islamophobia without disclosure that it is an internal ratio within already-classified content, not a rate against total discourse.
Additional Citations Tracked (1)
Scope: Incident tracker that acknowledges absent reliability testing and provides no baseline comparator
Several subsequent monitoring studies cite this report as establishing methodology. The absence of a discourse baseline and the unverified inter-coder agreement are not noted in these citations.