An alpha, graph-based Python network-intrusion-detection prototype. Its real-time components consume packet messages from Kafka, build and merge network graph state in Redis, dispatch node-analysis work, run model-based analysis, and print node results.

Status and non-reproducible limitations

Not an end-to-end runnable deployment as checked out. No packet producer/input-capture process or tested worker orchestration is provided. The included workers wait for messages on Kafka topics; starting them alone does not capture traffic.

The package metadata is alpha and only classifies Python 3.11. Its readme = "README.md" refers to a README missing from graph_based_intrusion_detection/, which may prevent standard package build metadata generation.
Node analysis loads models and scalers from hard-coded absolute paths outside this repository. Although similarly named model files exist under resources/models/, the configured paths do not point there. Update the configuration for an environment before starting the node analyzer.
Kafka and Redis addresses are hard-coded as localhost:9092 and localhost:6379 in the workers.
The local Compose stack uses unpinned latest container images and is not a reproducible production deployment.
No test suite, project test command, console entry point, or application deployment definition is present.

Do not use this prototype as a production intrusion-detection control without supplying and validating the missing capture, model-path, orchestration, retention, and security configuration.

Components and data flow

The real-time code uses these Kafka topics and Redis state:

packet_processor consumes packets, builds processed graph state, and publishes processed_states.
state_merger consumes processed_states and stores merged network state in Redis.
work_dispatcher reads network state, tracks nodes in Redis, sends work to node_analysis, and consumes node_analysis_results.
node_analyzer consumes node-analysis work, loads configured models/scalers, and publishes results.
network_state_visualizer reads the node registry from Redis and prints it to the terminal.

This describes the implemented connections, not a validated deployment order or a complete ingestion pipeline.

Prerequisites and setup status

The nested package at graph_based_intrusion_detection/ declares dependencies including Keras, TensorFlow, Kafka Python, Redis, NetworkX, NumPy, pandas, SciPy, scikit-learn, tqdm, and python-dotenv. Its package metadata identifies Python 3.11.

A standard package installation cannot be documented as reproducible until the missing nested README referenced by pyproject.toml is resolved. After package dependencies and metadata have been made installable, run the worker modules from graph_based_intrusion_detection/ as shown below. No dependency lockfile is provided.

Before starting node_analyzer, make the configured model and scaler paths in src/graph_based_intrusion_detection/utils/config.py valid for the runtime. Set LOG_FILE_PATH and LOG_LEVEL before starting workers; for example: export LOG_FILE_PATH=/tmp/network-event-detection.log and export LOG_LEVEL=INFO. No environment-based broker or Redis configuration is implemented.

Local infrastructure

From the nested package directory, start the provided local Kafka/Redis dependencies:

cd graph_based_intrusion_detection/deployment/kafka
docker compose up

This starts ZooKeeper, Kafka, and Redis and publishes Kafka on host port 9092 and Redis on host port 6379. It is local development infrastructure only; the Compose file has no authentication, persistent-volume, image-pinning, or application-worker service configuration. Keep Kafka, Redis, and every message producer isolated to a trusted network: workers unpickle message content and must never consume messages from untrusted publishers.

Worker commands

After dependencies are installed, models are configured, and Kafka/Redis are running locally, each worker has a direct Python module entry point. Run these from graph_based_intrusion_detection/; an optional trailing worker ID is accepted.

python -m graph_based_intrusion_detection.realtime_event_detection.packet_processor [worker-id]
python -m graph_based_intrusion_detection.realtime_event_detection.state_merger [worker-id]
python -m graph_based_intrusion_detection.realtime_event_detection.work_dispatcher [worker-id]
python -m graph_based_intrusion_detection.realtime_event_detection.node_analyzer [worker-id]
python -m graph_based_intrusion_detection.realtime_event_detection.network_state_visualizer [worker-id]

These are long-running workers. The repository does not establish a safe startup order, supervision model, shutdown procedure, health check, packet producer, or end-to-end validation command.

Development and testing

No tests or test runner are present. The notebooks under notebooks/ are exploratory artifacts rather than a documented operational workflow. There are no lint, format, build, or deployment scripts in the package metadata.

Repository layout

graph_based_intrusion_detection/src/graph_based_intrusion_detection/ — package source
- realtime_event_detection/ — Kafka/Redis workers
- packet_processing/, graph_processing/, analysis/ — graph construction and model analysis
- utils/config.py and utils/constants.py — hard-coded model paths, topics, ports, and runtime constants
graph_based_intrusion_detection/deployment/kafka/docker-compose.yml — local ZooKeeper, Kafka, and Redis stack
graph_based_intrusion_detection/resources/models/ — checked-in model/scaler artifacts (not the paths currently configured for node analysis)
graph_based_intrusion_detection/notebooks/ — exploratory notebooks
infosheet.md — capture/device notes, not operating documentation