Cyolo SecOps Middleware

A lightweight middleware service that forwards Cyolo IDAC security events to Google Security Operations (SecOps/Chronicle) using the official Google SecOps SDK and UDM (Unified Data Model) format.

Architecture

┌─────────────┐     Syslog (CEF/JSON)     ┌──────────────────┐
│   Cyolo     │ ────────────────────────► │   Middleware     │
│   IDAC      │      UDP Port 514         │   (Docker)       │
└─────────────┘                           └────────┬─────────┘
                                                  │
                                                  │ Google SecOps SDK
                                                  │ (ingest_udm)
                                                  ▼
                                         ┌──────────────────┐
                                         │  Google SecOps   │
                                         │  Ingestion API   │
                                         └──────────────────┘

Features

• Official SDK: Uses Google's official secops Python SDK for reliable integration
• CEF and JSON parsing: Supports both CEF and JSON syslog formats from Cyolo
• UDM mapping: Converts 200+ Cyolo event types to Google SecOps UDM format
• Intelligent batching: Size-based and time-based batching for efficient API usage
• Retry logic: Built-in retry with exponential backoff
• Health checks: HTTP endpoint for container orchestration
• Structured logging: JSON-formatted logs for observability

Quick Start

Prerequisites

1. GCP Project linked to your Google SecOps instance
2. IAM Role: Service account with Chronicle API Admin (roles/chronicle.admin) or Chronicle API Editor role
3. Authentication - One of:
   • Application Default Credentials (ADC) via gcloud auth application-default login
   • Service Account Key file
   • GCP metadata service (when running on GCP)
4. Cyolo IDAC configured with syslog export
5. Docker and Docker Compose

1. Setup Authentication

The middleware uses the official Google SecOps SDK which supports multiple authentication methods:

Option A: Application Default Credentials (Recommended for Development)

# Login with gcloud - simplest for local development
gcloud auth application-default login

The SDK will automatically use these credentials.

Option B: Service Account Key

# 1. Create a service account
gcloud iam service-accounts create cyolo-secops-ingestion \
  --display-name="Cyolo SecOps Ingestion"

# 2. Grant the Chronicle API Admin role
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:cyolo-secops-ingestion@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/chronicle.admin"

# 3. Download credentials key
gcloud iam service-accounts keys create credentials.json \
  --iam-account=cyolo-secops-ingestion@YOUR_PROJECT_ID.iam.gserviceaccount.com

Then set GOOGLE_APPLICATION_CREDENTIALS to point to the key file.

Option C: Workload Identity Federation (Production)

For production deployments, use Workload Identity Federation for keyless authentication. See GCP WIF documentation.

Required Information

You'll need:

• Project ID: Your GCP project ID (linked to SecOps)
• Instance ID: Your SecOps instance UUID (also called customer_id, found in SecOps settings)
• Region: Your SecOps instance region (e.g., us, europe)

2. Configure Environment

Create a .env file:

# Required - GCP/SecOps Configuration
SECOPS_PROJECT_ID=your-gcp-project-id
SECOPS_INSTANCE_ID=your-secops-instance-uuid  # UUID format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
SECOPS_REGION=us  # or europe, asia-southeast1, etc.

# Optional - Authentication (if not using ADC)
GOOGLE_APPLICATION_CREDENTIALS=/app/credentials.json

# Optional - Tuning
LOG_LEVEL=INFO
SYSLOG_FORMAT=cef  # or json

3. Start the Middleware

docker-compose up -d

4. Verify Connection

# Check health
curl http://localhost:8080/health

# Verify SecOps connection
curl http://localhost:8080/verify

# View stats
curl http://localhost:8080/stats

5. Configure Cyolo

Point Cyolo's syslog export to the middleware:

Via Cyolo Admin Portal:

1. Go to Settings → Syslog Servers
2. Click Add Syslog Server
3. Enter the middleware host IP and port 514
4. Select CEF format (recommended) or JSON
5. Enable for all sites or specific sites

6. Test with a Sample Event

# Send a test CEF event
echo '<134>Feb 2 13:00:00 test CEF:0|cyolo|idac|1.0|101110013|User Login|1|act=login dst=10.1.2.3 [email protected]' | nc -u localhost 514

# Wait for batch flush (5 seconds)
sleep 6

# Check logs for result
docker logs cyolo-secops-middleware 2>&1 | tail -20

Configuration

Environment Variables

Supported Regions

See Google SecOps regions for the complete list.

Event Mapping

The middleware maps Cyolo events to Google SecOps UDM event types:

HTTP Endpoints

Development

Setup

# Install dependencies
poetry install

# Run tests
poetry run pytest

# Format code
poetry run black src/ tests/
poetry run ruff src/ tests/

# Type checking
poetry run mypy src/

Running Locally

# Set required environment variables
export SECOPS_PROJECT_ID=your-project-id
export SECOPS_INSTANCE_ID=your-instance-uuid

# Authenticate (simplest method for local dev)
gcloud auth application-default login

# Run the middleware
poetry run python -m cyolo_secops.main

Project Structure

cyolo-secops-middleware/
├── src/cyolo_secops/
│   ├── main.py              # Entry point
│   ├── config.py            # Pydantic settings
│   ├── syslog_receiver.py   # UDP syslog server
│   ├── batcher.py           # Event batching
│   ├── secops_client.py     # Google SecOps SDK wrapper
│   ├── models.py            # Data models
│   ├── parsers/
│   │   ├── cef_parser.py    # CEF format parser
│   │   └── json_parser.py   # JSON format parser
│   └── mapper/
│       ├── udm_mapper.py    # UDM mapping logic
│       └── event_types.py   # Event type mappings
├── tests/
├── Dockerfile
├── docker-compose.yml
└── pyproject.toml

Troubleshooting

Common Issues

"Authentication failed"

• Ensure you're authenticated: gcloud auth application-default login
• Or set GOOGLE_APPLICATION_CREDENTIALS to a valid service account key
• Verify the service account has Chronicle API Admin or Chronicle API Editor role

"API error: 403"

• Verify the service account has the required IAM role on the GCP project
• Check that the GCP project is linked to your SecOps instance
• Verify SECOPS_PROJECT_ID and SECOPS_INSTANCE_ID are correct

"API error: 400"

• Check the logs for UDM validation errors
• Ensure events have required fields (timestamp, event_type, principal)

Events not appearing in SecOps

• Check that Cyolo is sending to the correct syslog address
• Verify the syslog format matches (cef vs json)
• Check middleware logs for parsing errors
• Use /stats endpoint to see processing counts

Debug Mode

LOG_LEVEL=DEBUG docker-compose up

Security Considerations

1. Credentials: Store GCP credentials securely, never commit to source control
2. Network: Run the middleware in an isolated network segment
3. Ports: The syslog port (514) should only be accessible from Cyolo
4. Updates: Keep the middleware updated for security patches

License

Copyright (c) Cyolo Security. All rights reserved.