ML Detection

Tamandua EDR includes a Malware-SMELL (Similarity-based Malware Embedding

for Lifelong Learning) architecture for ML-assisted malware detection. In the

current alpha, ML is an integration and validation track, not a production

blocking claim.

Validation Status

The ML architecture and service pipeline are implemented, but production-grade

ML claims require benchmark evidence. The current checkpoint is smoke-scale and

the generated model card is explicitly not_production_ready.

Internally, the governed execution path is tracked by the ML execution master

handoff and benchmark critical path under docs/benchmarks/runs/. Those

artifacts currently mark Wave 1 acquisition as validation-ready only and keep

ML-1 through ML-6 blocked until the real lab acquisition, sanitized dataset

manifest, candidate training, ONNX parity, service benchmark, replay, and

holdout evidence exist.

The ML-1 through ML-6 operator summaries now publish validation-only commands,

guarded execution commands, authorization hashes, and two-step operator

sequences, but those are authorization controls only. They are not model-quality

or production detection evidence.

Current internal evidence:

Treat ML output as one detection signal. Do not treat it as an autonomous

blocking verdict until false-positive review, shadow/canary rollout, rollback,

and end-to-end benchmark gates pass.

Architecture Overview

+------------------+     +------------------+     +------------------+
|   Binary Input   | --> |  Binary-to-Image | --> |   VGG-19 Encoder |
|   (PE/ELF/Mach-O)|     |   Conversion     |     |   (Pretrained)   |
+------------------+     +------------------+     +--------+---------+
                                                          |
                                                          v
+------------------+     +------------------+     +------------------+
|   Classification | <-- |  S-Space Vector  | <-- |  Latent Space    |
|   (KNN)          |     |  (Similarity)    |     |  (256-dim)       |
+------------------+     +------------------+     +------------------+
                                |
                                v
                         +--------------+
                         |   Markers    |
                         | (Benign/Mal) |
                         +--------------+

Model Components

How It Works

1. Binary-to-Image Conversion

The first step converts a binary file into a visual representation:

# Conceptual process
raw_bytes = read_binary_file(path)
normalized = normalize_bytes(raw_bytes)  # Scale to 0-1
image = reshape_to_grid(normalized, size=64)  # 64x64 grayscale

This transformation preserves structural patterns in binaries while enabling the use of computer vision techniques.

2. Feature Extraction

The VGG-19 encoder extracts high-level features from the binary image:

• Input: 64x64 grayscale image
• Backbone: VGG-19 convolutional layers (pretrained on ImageNet)
• Custom Head: Fully connected layers for 256-dim latent space
• Output: Latent vector representing the binary's characteristics

3. Similarity Space (S-Space)

The S-Space computes similarity scores using Cauchy distribution:

S(z_i, z_j) = 1 / (1 + ||z_i - z_j||^2)

Where z_i and z_j are latent vectors. This produces a similarity score between 0 and 1.

4. Marker-Based Classification

Markers are reference points in the latent space:

The S-vector is computed as similarities to all markers, creating a 5-dimensional representation for final classification.

5. Classification

A K-Nearest Neighbors (KNN) classifier makes the final decision based on S-vectors of known samples:

• Compare unknown sample's S-vector to reference dataset
• Classify based on majority vote of K nearest neighbors
• Return confidence score based on neighbor distances

Zero-Shot Detection Roadmap

The Malware-SMELL design targets zero-shot learning, but Tamandua does not

currently claim validated zero-day or cross-time robustness. Those claims require

Lane ML-6 holdout results over post-cutoff sources such as MalwareBazaar,

selected VX Underground InTheWild archives after governance review, VirusShare,

and external goodware.

The intended mechanism is:

1. The model learns what benign software looks like
2. Malware deviates from benign patterns
3. New malware naturally falls into dissimilarity regions

Confidence Scores

The ML model returns confidence scores with explanations:

Score Interpretation

Confidence Factors

The confidence score is influenced by:

1. Distance to markers: How close the sample is to reference markers
2. Neighborhood consistency: Agreement among nearest neighbors
3. Feature clarity: Distinctiveness of extracted features
4. Known family similarity: Similarity to known malware families

API Usage

Predict Endpoint

# Analyze a file
curl -X POST "https://api.tamandua.io/v1/ml/predict" \
  -H "Authorization: Bearer $TOKEN" \
  -F "file=@suspicious.exe"

Response:

{
  "prediction": "malicious",
  "confidence": 94.7,
  "family": "trojan.generic",
  "features": {
    "entropy": 7.2,
    "import_hash": "abc123...",
    "pe_sections": 5
  },
  "explanation": {
    "top_indicators": [
      "High similarity to known RAT patterns",
      "Unusual import table structure",
      "Encrypted section detected"
    ],
    "marker_distances": {
      "benign_1": 0.12,
      "benign_2": 0.15,
      "benign_3": 0.18,
      "malicious_1": 0.92,
      "malicious_2": 0.87
    }
  }
}

Batch Prediction

# Analyze multiple files
curl -X POST "https://api.tamandua.io/v1/ml/predict/batch" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "hashes": ["sha256_1", "sha256_2", "sha256_3"],
    "include_explanation": true
  }'

Model Information

# Get model metadata
curl "https://api.tamandua.io/v1/ml/model/info" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "model_version": "smoke-or-candidate-id",
  "encoder": "VGG-19",
  "latent_dim": 256,
  "training_samples": 0,
  "model_status": "loaded_untrained_or_smoke",
  "quality_gate": "not_production_ready",
  "last_updated": "example"
}

Production accuracy, cross-time recall, and endpoint impact must come from the

published ML benchmark reports, not from this shape example.

Model Updates

Update Process

Model update cadence is a rollout target, not a current production guarantee.

Before update automation is enabled, Tamandua needs signed model bundles,

ONNX/agent parity, native ONNX Runtime provisioning, canary/shadow evidence, and

rollback proof.

Enabling Auto-Updates

# Roadmap configuration shape; not a production-ready default.
ml:
  auto_update: true
  update_channel: "stable"  # stable, beta, canary
  update_schedule: "weekly"

Manual Updates

# Force model update
tamanduactl ml update

# Check for updates
tamanduactl ml check-updates

# Rollback to previous version
tamanduactl ml rollback

False Positive Handling

Common False Positive Causes

Reducing False Positives

1. Allowlist by Hash

   tamanduactl allowlist add --hash sha256:abc123...
   

1. Allowlist by Signer

   tamanduactl allowlist add --signer "Company Name"
   

1. Allowlist by Path

   tamanduactl allowlist add --path "/opt/internal/*"
   

1. Adjust Confidence Threshold

   ml:
     alert_threshold: 80  # Only alert on 80%+ confidence
     block_threshold: 95  # Only block on 95%+ confidence
   

Submitting False Positives

# Report a false positive
tamanduactl submit-fp --hash sha256:abc123... \
  --reason "Internal tool, legitimate" \
  --evidence "Signed by our company"

False positive submissions are reviewed by the Tamandua security team and used to improve the model.

Performance Considerations

Inference Times

Resource Usage

Optimization Options

# Performance tuning
ml:
  # Use GPU if available
  device: "auto"  # auto, cuda, cpu

  # Batch similar files
  batch_size: 8

  # Skip large files
  max_file_size: 100MB

  # Cache predictions
  cache_enabled: true
  cache_ttl: 3600

  # Pre-filter with YARA
  prefilter_enabled: true

Explainability

Tamandua provides explainability features to understand ML decisions:

Feature Importance

# Get detailed explanation
curl "https://api.tamandua.io/v1/ml/explain/sha256:abc123" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "prediction": "malicious",
  "confidence": 92.5,
  "feature_importance": [
    {"feature": "imports_suspicious", "weight": 0.35},
    {"feature": "entropy_sections", "weight": 0.28},
    {"feature": "packer_detected", "weight": 0.22},
    {"feature": "string_patterns", "weight": 0.15}
  ],
  "similar_samples": [
    {"hash": "def456...", "family": "emotet", "similarity": 0.94},
    {"hash": "ghi789...", "family": "emotet", "similarity": 0.91}
  ]
}

Visualization

The dashboard provides visual explanations:

• Heatmaps: Highlighting suspicious binary regions
• Similarity Maps: Showing proximity to known samples
• Feature Charts: Comparing to benign/malicious baselines

Drift Detection

The ML service monitors for concept drift to ensure continued accuracy:

When drift is detected, administrators are notified to review model performance.

Related Documentation

• Detection Overview - Multi-layered detection strategy
• YARA Rules - Signature-based detection
• Response Actions - Automated response to ML detections