Terrorist Organizations - Functional Guide¶

This guide explains how to use the terrorist organizations use case for building knowledge graphs and running evaluations.

Prerequisites¶

1. FalkorDB running: redis-cli ping should return PONG
2. OpenAI API key: Set OPENAI_API_KEY environment variable
3. Data downloaded: FTM files in use_cases/terrorist_orgs/data/

Building the Knowledge Graph¶

Step 1: Load the Ontology¶

First, load the FollowTheMoney ontology into a graph:

aletheia build-ontology-graph \
  --use-case terrorist_orgs \
  --knowledge-graph terrorist_orgs_ontology

This creates nodes for: - Entity types (Organization, Person, Sanction, etc.) - Relationship types (SANCTION, HAS_ALIAS, etc.)

Step 2: Build the Knowledge Graph¶

aletheia build-knowledge-graph \
  --use-case terrorist_orgs \
  --knowledge-graph terrorist_orgs \
  --schema-mode graph-hybrid \
  --ontology-graph terrorist_orgs_ontology

Options:

Step 3: Verify the Graph¶

# Show graph statistics
aletheia show-graph --knowledge-graph terrorist_orgs

Or query directly:

# Count by entity type
redis-cli GRAPH.QUERY terrorist_orgs \
  "MATCH (n) RETURN labels(n) as type, count(*) ORDER BY count(*) DESC"

# Count relationships
redis-cli GRAPH.QUERY terrorist_orgs \
  "MATCH ()-[r]->() RETURN type(r), count(*) ORDER BY count(*) DESC"

Running Evaluations¶

Evaluation Datasets¶

Curated Dataset (Recommended)¶

The curated dataset focuses on questions that GraphRAG excels at:

Run Evaluation¶

# Curated dataset (recommended)
aletheia evaluate-ragas \
  --knowledge-graph terrorist_orgs \
  --questions use_cases/terrorist_orgs/evaluation_questions_curated.json \
  --grounding-mode strict

# Full dataset
aletheia evaluate-ragas \
  --knowledge-graph terrorist_orgs \
  --questions use_cases/terrorist_orgs/evaluation_questions.json \
  --grounding-mode strict

# Compare with baseline (cosine-only vs BFS+cosine)
aletheia evaluate-ragas \
  --knowledge-graph terrorist_orgs \
  --questions use_cases/terrorist_orgs/evaluation_questions_curated.json \
  --compare-baseline

Expected Scores¶

The curated dataset better reflects GraphRAG capabilities without penalizing for SQL-like queries.

Query Examples¶

Using Aletheia MCP Server¶

With Claude Desktop and the Aletheia MCP server configured:

User: "What authorities have designated Hamas?"
Claude: Hamas is designated by the US State Department, UK Home Office,
        and Australia Home Affairs.

Using Cypher (FalkorDB)¶

-- Find organizations designated by multiple authorities
MATCH (o:Organization)
WHERE o.summary CONTAINS 'US State Department'
  AND o.summary CONTAINS 'UK Home Office'
RETURN o.name, o.summary
LIMIT 10

-- Find all aliases for an organization
MATCH (o:Organization)-[:HAS_ALIAS]->(a)
WHERE o.name CONTAINS 'Hamas'
RETURN o.name, collect(a.name) as aliases

-- List designating authorities
MATCH (pb:PublicBody)
RETURN pb.name, pb.summary

Question Types Explained¶

Alias Lookup¶

Test the graph's ability to resolve alternative names:

{
  "question": "What alias is used for al-Shabaab?",
  "answer": "al-Hijra"
}

Entity Existence¶

Verify if entities exist with specific attributes:

{
  "question": "Is Hamas designated as a Foreign Terrorist Organization?",
  "answer": "Yes, Hamas is designated as an FTO by the US State Department"
}

Multi-hop Reasoning¶

Questions requiring graph traversal:

{
  "question": "What is the relationship between AQIM and GSPC?",
  "answer": "AQIM evolved from the Salafist Group for Call and Combat (GSPC)"
}

Cross-Jurisdiction¶

Questions spanning multiple authorities:

{
  "question": "What authorities have designated Hezbollah?",
  "answer": "US State Department, UK Home Office, and Australia"
}

Excluded Questions¶

The curated dataset excludes questions that GraphRAG doesn't handle well:

These questions may work but produce lower scores due to the nature of semantic search.

Troubleshooting¶

Low Recall Scores¶

If context recall is low:

1. Increase BFS depth in search config
2. Check that entities were properly ingested
3. Verify relationships exist between entities

Missing Aliases¶

If alias queries fail:

1. Check HAS_ALIAS relationships exist:

   redis-cli GRAPH.QUERY terrorist_orgs \
     "MATCH ()-[r:HAS_ALIAS]->() RETURN count(r)"
   

2. Ensure ontology-first or graph-hybrid mode was used (includes HAS_ALIAS)

Grounding Failures¶

If grounding verification fails:

1. Use --grounding-mode lenient to see what's happening
2. Check if answers contain entities not in evidence
3. Review the grounding report in output JSON