Aviation Safety - Functional Guide¶

This guide explains how to use the aviation safety use case for building knowledge graphs and running evaluations.

Prerequisites¶

FalkorDB running: redis-cli ping should return PONG
OpenAI API key: Set OPENAI_API_KEY environment variable
Data files: Markdown incident reports in use_cases/aviation_safety/data/

Building the Knowledge Graph¶

Step 1: Load the Ontology (Optional)¶

For better schema inference, load the aviation ontology:

aletheia build-ontology-graph \
  --use-case aviation_safety \
  --knowledge-graph aviation_safety_ontology

Step 2: Build the Knowledge Graph¶

aletheia build-knowledge-graph \
  --use-case aviation_safety \
  --knowledge-graph aviation_safety \
  --schema-mode graph-hybrid \
  --ontology-graph aviation_safety_ontology

Options:

Option	Description
`--schema-mode graph-hybrid`	Use semantic alignment with aviation ontology
`--schema-mode llm`	Auto-discover schema from data
`--build-communities`	Create community clusters
`--reset`	Clear existing graph before building

Step 3: Verify the Graph¶

# Show graph statistics
aletheia show-graph --knowledge-graph aviation_safety

Or query directly:

# Count by entity type
redis-cli GRAPH.QUERY aviation_safety \
  "MATCH (n) RETURN labels(n) as type, count(*) ORDER BY count(*) DESC"

Running Evaluations¶

Evaluation Datasets¶

Dataset	File	Questions	Description
Curated	`evaluation_questions_curated.json`	35	Optimized for GraphRAG
Full	`evaluation_questions.json`	50	All question types

Question Types¶

Type	Description	Example
`cause_lookup`	Find primary cause	"What caused incident 2024-0157-EU?"
`contributing_factor_lookup`	Find contributing factors	"What factors contributed to the Madrid incident?"
`aircraft_lookup`	Find aircraft type	"What aircraft was involved in the Paris incident?"
`operator_lookup`	Find airline/operator	"Which airline operated the Frankfurt aircraft?"
`incident_at_location`	Find incident at location	"What happened at Nice Airport?"
`entity_existence`	Verify specific facts	"What bird species was involved?"
`entity_description`	Describe incident/entity	"Describe the Amsterdam bird strike"
`recommendation_lookup`	Find safety recommendations	"What recommendations came from this incident?"

Run Evaluation¶

# Curated dataset (recommended)
aletheia evaluate-ragas \
  --knowledge-graph aviation_safety \
  --questions use_cases/aviation_safety/evaluation_questions_curated.json \
  --grounding-mode strict

# Full dataset
aletheia evaluate-ragas \
  --knowledge-graph aviation_safety \
  --questions use_cases/aviation_safety/evaluation_questions.json \
  --grounding-mode strict

# Compare with baseline
aletheia evaluate-ragas \
  --knowledge-graph aviation_safety \
  --questions use_cases/aviation_safety/evaluation_questions_curated.json \
  --compare-baseline

Expected Scores¶

Dataset	Context Precision	Context Recall	Faithfulness	Answer Similarity
Curated (35q)	>0.5	>0.6	>0.7	>0.6
Full (50q)	~0.4	~0.5	~0.6	~0.5

Query Examples¶

Using Aletheia MCP Server¶

With Claude Desktop and the Aletheia MCP server:

User: "What caused the turbulence incident near Barcelona?"
Claude: The incident was caused by unpredicted clear air turbulence
        associated with a jetstream boundary.

Using Cypher (FalkorDB)¶

-- Find all incidents by operator
MATCH (op:Operator)-[:HAS_OPERATOR]-(o:Occurrence)
WHERE op.name CONTAINS 'Air France'
RETURN op.name, o.name, o.summary

-- Find incidents by aircraft manufacturer
MATCH (m:Manufacturer)<-[:MANUFACTURED_BY]-(a:Aircraft)<-[:HAS_AIRCRAFT]-(o:Occurrence)
RETURN m.name as manufacturer, count(o) as incidents

-- Trace full incident path
MATCH path = (m:Manufacturer)<-[:MANUFACTURED_BY]-(a:Aircraft)<-[:HAS_AIRCRAFT]-(o:Occurrence)
  -[:HAS_AIRPORT]->(ap:Airport)-[:LOCATED_IN]->(c:Country)
RETURN path

Incident Data Format¶

Each incident is stored as a markdown file with structured sections:

# Incident Report 2024-0157-EU

## Metadata
- **Incident ID**: 2024-0157-EU
- **Date**: 2024-02-15
- **Location**: Paris Charles de Gaulle (LFPG)
- **Country**: France
- **Flight Phase**: Landing

## Aircraft
- **Type**: Airbus A330-200
- **Registration**: F-GZCA
- **Operator**: Air France

## Incident Description
During approach to runway 26R, the crew received a hydraulic
system warning...

## Outcome
- **Injuries**: None
- **Aircraft Damage**: Minor

## Findings
### Primary Cause
Hydraulic pump failure due to manufacturing defect

### Contributing Factors
- Delayed maintenance action
- Limited crew training on this scenario

Adding New Incidents¶

Create markdown file in data/ directory following the format above
Ensure all sections are present

Rebuild graph:

aletheia build-knowledge-graph \
  --use-case aviation_safety \
  --knowledge-graph aviation_safety \
  --reset

ECCAIRS Taxonomy¶

To use the ECCAIRS taxonomy:

Download ECCAIRS XML from EASA

Convert to TTL:

python use_cases/aviation_safety/eccairs_taxonomy/eccairs_parser.py \
  eccairs_taxonomy/Eccairs_Aviation.xml \
  -o ontology/eccairs_aviation.ttl

Load ontology and rebuild graph

Troubleshooting¶

Missing Entities¶

If entities aren't extracted:

Check markdown format matches expected structure
Verify sections have proper headers (##)
Look for parser errors in output

Low Recall¶

If relevant incidents aren't found:

Increase search limit
Check entity names match between questions and data
Verify relationships were created

Poor Faithfulness¶

If answers aren't grounded:

Use --grounding-mode strict
Check if questions require data not in incidents
Review grounding report for rejection reasons