CCYBEROPS
System Architecture · source of truth

CyberOps Platform Architecture

A unified, multi-tenant, AI-assisted security-operations platform. It turns asset discovery, vulnerability scanning, AI-assisted penetration testing, OSINT/threat analysis, and compliance mapping into a single correlated view of organizational risk — with continuous retesting, across an organization's authorized attack surface.

Build depth comprehensive Scale tier large · 50k+ concurrent Monorepo npm workspaces API NestJS 11 · REST /v1 Web Next.js · App Router DB PostgreSQL 18 · RLS

§1 System Overview

Core problem → solution: fragmented security tooling → one workflow that discovers, assesses, validates, correlates, prioritizes, and retests.

Five key workflows (SPEC §5)

W1 · AI Pentest

AI-powered automated penetration test — multi-agent planner produces attack paths + evidence.

W2 · Discovery & VA

Asset discovery & vulnerability assessment across the authorized surface.

W3 · OSINT

Threat & OSINT investigation — correlate indicators against assets and findings.

W4 · Compliance

Compliance & security-posture assessment mapped to control frameworks.

W5 · Retest

Finding remediation & continuous retesting with a tamper-evident trail.

⚠ Safety boundary (binding, RESEARCH §5)

The platform runs authorized offensive tooling only. It ships real safe recon (DNS / HTTP-fingerprint / TLS) and a template-driven finding engine, and models exploit-validation as state — unvalidated → validating → validated | failed — gated behind verified authorization scope + human approval. It does not ship a live-exploitation module that fires payloads at arbitrary targets. Enforced by the authorization-scope, SSRF, rate/concurrency, and agent-sandbox invariants (§9).

§2 Architecture Layers & Data Flow

Topology (monorepo)

cyberops-2/ · npm workspaces monorepo apps/apiNestJS 11 — REST /v1, WS gateway, BullMQ workers apps/webNext.js App Router — marketing + app shell packages/sharedZod schemas + DTO/enum types (contract source) scripts/invariant-lint.mjs · smoke-test.sh · seed docker-compose.yml → api · web · postgres · redis · worker · .env.example Precedence: RESEARCH > ARCHITECTURE > SPEC

Request path (synchronous)

Browser→ Next.js SSR + TanStack Query→ API /v1→ JwtAuthGuard→ TenantGuard→ RolesGuard→ ZodValidationPipe→ Controller→ Service→ TenantScopedRepository→ Postgres (RLS)

Every mutation emits an audit_events row and — when it must reach a second system — an outbox_events row in the same transaction.

Three runtime paths

SYNC Browser API + Guards Service+Repo Postgres RLS ASYNC Service BullMQ · Redis Worker ScannerAdapters queues: scan · outbox · email · webhook · pentest Finding stream → persist + WS Outbox relay search index · email (Resend) · webhooks (HMAC) REALTIME Worker Socket.io + Redis adapter room scan:<id>
Web / client API / guards Queue / worker Database

Data flow across workflows

Asset (W2)→ Scan→ Finding CVSS/CWE→ + Threat intel (W3)→ Compliance (W4)→ Dashboard→ Remediate + Retest (W5)

AI pentest (W1) consumes assets/findings and produces attack paths + evidence.

§3 Backend Modules (NestJS · domain-per-module)

One folder per domain under apps/api/src/modules/*.

auth

Register / login / logout, JWT httpOnly-cookie session, password reset, MFA-ready.

User · Session · PasswordResetToken

tenancy

Orgs, memberships, RBAC roles, tenant-scope chokepoint, RLS context.

Organization · Membership · Role

assets

Targets, assets, hosts, services, technologies, authorization scope.

Asset · Host · Service · AuthorizationScope

scans

Scan runs, adapter orchestration, BullMQ, WS progress.

ScanRun · ScanResult

findings

Findings, CVSS/CWE, severity, correlation, validation state, retest.

Finding · FindingValidation · Retest

pentest

AI pentest engine (multi-agent planner), attack paths, evidence, approval gate.

PentestRun · AttackPath · Evidence · ApprovalRequest

osint

Indicators, STIX/TAXII threat feeds, correlation.

Indicator · ThreatFeed · Investigation

compliance

Frameworks, controls, mappings, posture, evidence.

Framework · Control · ControlMapping · PostureSnapshot

reports

PDF / JSON / CSV export of findings / compliance / pentest.

Report

alerts

Email outbox + outbound webhooks (guaranteed delivery).

PendingEmail · PendingWebhook · WebhookEndpoint

audit

Append-only, hash-chained audit log.

AuditEvent

outbox

Transactional-outbox relay to second systems.

OutboxEvent

Cross-cutting common/

§4 Data Layer

PostgreSQL 18 · TypeORM

  • PKs = uuidv7() — timestamp-ordered, index-friendly at insert scale.
  • Finding payloads / evidence = jsonb; enums as Postgres enum.
  • Reversible DataSource migrations (up/down); seed for roles, severities, frameworks, demo org.

Tenant isolation (defense in depth)

  • Every tenant-owned table carries tenant_id.
  • Postgres RLS scopes by a session app.tenant_id GUC set per request/job.
  • TenantScopedRepository is the single chokepoint that sets the GUC and filters.
  • Object-level assertResourceAccess on every user-owned resource and its sub-resources.

Idempotency & audit chain

  • inbound_webhook_events UNIQUE (provider, event_id).
  • pending_emails UNIQUE (to_address, template_key, business_ref_id).
  • Audit: hash = sha256(prev_hash || canonical_json(payload)) — append-only, no update/delete path.

Pooling & scale

  • pg pool tuned; PgBouncer (transaction mode) documented for prod.
  • Hot tables findings, scan_results indexed on (tenant_id, severity, created_at).
  • Read-replica split documented as a cloud-scale step.

§5 Scanner & Adapter Engine

ScanOrchestrator (BullMQ worker) selects adapters by profile → streams RawFindings → normalizes to Finding (CVSS/CWE/severity) → persists → emits WS progress.

ScanOrchestrator TargetResolverresolveAndValidate() — INV-2 adapter.run(target) RawFinding → SSRF guard runs BEFORE every adapter network action: rejects private/link-local/reserved, requires active verified AuthorizationScope, pins resolved IP (anti-DNS-rebinding). normalize → persist → WS progress

Built-in safe adapters always available

  • DnsReconAdapter — records + bounded subdomain wordlist (authorized zones only).
  • HttpFingerprintAdapter — headers, tech/version, missing security headers, cookie flags, safe .git/.env probes.
  • TlsInspectAdapter — cert chain, expiry, weak protocol/cipher.
  • TemplateEngineAdapter — Nuclei-style YAML signatures (offline corpus) → CVE/CWE-tagged findings.

External adapters degrade gracefully

  • NucleiAdapter — spawns nuclei -jsonl, parses JSONL to findings.
  • TrivyAdapter — spawns trivy … --format json for repo/IaC/SBOM findings.

Used only when the binary is present; the platform stays fully functional on the built-in safe set alone.

◆ ScannerAdapter interface

{ id, kind, capabilities, isAvailable(): Promise<boolean>, run(target, opts, ctx): AsyncIterable<RawFinding> } — pluggable, testable, safe-by-construction.

§6 AI Pentest Orchestration (multi-agent)

PentestPlanner decomposes a run into phases following the VulnBot/AutoPentest/xOffense multi-agent pattern. Each phase is an agent role with an allowlisted tool set that calls only the safe scanner adapters.

reconsafe discovery analysisreason attack paths validationapproval-gated reportattack paths + evidence ⚑ ApprovalRequest (human-in-the-loop) State-changing validation → ApprovalRequest + active-scope re-check. No live exploitation is ever performed.

LLM provider & prompt-injection

  • LlmProvider interface — provider-abstracted; Claude Messages API default.
  • All target-derived text (banners, HTTP bodies, OSINT) is untrusted — passed as data, never as instructions (mitigation R7).

Offline / degraded mode

  • With no LLM key, the planner falls back to a deterministic rule-based phase engine.
  • A pentest run stays demonstrably functional without external AI — findings + attack paths from the safe adapters.
  • Output: AttackPath (MITRE ATT&CK + CWE), Evidence (encrypted at rest), risk-scored Findings.

§7 Realtime, Async, Notifications & Observability

WebSocket

Socket.io gateway + @socket.io/redis-adapter for multi-instance fan-out. Auth: JWT on connection, Origin allowlist, per-connection message-rate budget, channel authorization (a client may only join rooms for its own tenant's scans).

Queues

BullMQ v5 on Redis (maxmemory-policy=noeviction, dedicated logical DB). Queues: scan outbox email webhook pentest. Idempotent jobs; DLQ after capped backoff.

Outbox relay

Drains outbox_events → search index (Postgres FTS in-cluster; Algolia/OpenSearch documented swap), email, outbound webhooks (HMAC signed at insert, Idempotency-Key per send, ≤5 attempts).

Email & telemetry

Resend v6 via the email queue with rate limiting — never inline in a request handler. Sentry v10 (instrument.ts first), structured JSON logs (pino), Prometheus-style /metrics + /api/health.

§8 Frontend (Next.js · App Router)

(marketing)Landing — nav, hero, features, how-it-works, pricing, footer. Ported verbatim from DESIGN-TEMPLATE.html.
(app)Authenticated shell — aside sidebar + topbar + glass content. Dashboard, assets, scans, findings, pentest, OSINT, compliance, reports, audit, settings.
(auth)Login / register / reset.

Data

  • TanStack Query v5 (HydrationBoundary + per-request QueryClient).
  • Typed API client generated from packages/shared Zod schemas.
  • WS client for live scan progress.

Design system

  • Design tokens copied verbatim from the template :root into apps/web/app/globals.css.
  • Dark-first with a light variant + runtime theme toggle + prefers-color-scheme.
  • Every key workflow completable through the UI, including terminal steps (review / approve / export).

↳ A static mockup of every one of these screens ships alongside this document as index.html.

§9 Architectural Invariants

Machine-checkable form in invariants.json; enforced by npm run lint:invariants (exit-code). Amending one requires updating §9 + invariants.json + an ADR in the same commit.

IDRuleCheck type
INV-1All scoped DB access goes through TenantScopedRepository; no ad-hoc getRepository( outside the tenancy chokepoint.forbidden-pattern
INV-2SSRF: the orchestrator calls resolveAndValidate before dispatching any adapter.run(.boundary-order
INV-3Authorization scope re-validated before any network action of a scan/pentest job (deny-by-default).manual
INV-4Inbound webhook receiver verifies signature before any DB read/write.boundary-order
INV-5Audit log is append-only — no update/delete/remove on audit events.forbidden-pattern
INV-6Rate-limit config (Redis-backed throttler) is present and wired.required-file
INV-7No hardcoded secrets (AWS keys, sk_live_, private keys) in source.forbidden-pattern
INV-8Inbound webhook idempotency: UNIQUE (provider, event_id).required-unique
INV-9Email idempotency: UNIQUE (to_address, template_key, business_ref_id).required-unique
INV-10Design token layer exists (apps/web/app/globals.css) with template tokens.required-file
INV-11Health endpoint present.required-file
INV-12UI coverage: every screen routes, every non-internal endpoint referenced by UI, every workflow has an e2e test.ui-coverage
INV-13Object-level authorization (assertResourceAccess) on user-owned resources and sub-resources.manual
INV-15assertResourceAccess invoked in every owner-bearing service (machine-checked call-site).required-pattern
INV-14Header buffers sized for accumulated cookies (--max-http-header-size / proxy config).manual

§10 Dependencies (pinned)

API

@nestjs/*^11 · @nestjs/throttler^6 · websockets + platform-socket.io · @socket.io/redis-adapter · typeorm^0.3 · pg · bullmq^5 · ioredis · zod · bcrypt · @nestjs/jwt · resend^6 · @sentry/nestjs^10 · pino

Web

next^15 · react^18 · @tanstack/react-query^5 · tailwindcss^3 · shadcn/ui (vendored) · socket.io-client · zod

Shared / dev

zod (single contract source) · jest + supertest (API) · vitest / @playwright/test (web e2e) · typescript · eslint

◆ Build-environment version note (ADR-006)

RESEARCH verified the bleeding edge (Next 16 / Tailwind v4 / TypeORM 1.1). Where a bleeding-edge major introduced install/build instability in the sandbox, the build pins the nearest battle-tested major (Next 15, Tailwind v3, TypeORM 0.3) and logs the substitution as an ADR — the deliverable (stack, design system, workflows) is preserved; only the pin moves. A build-environment substitution, not a stack change.

§11 Key Technical Decisions

DecisionOverWhy
Monorepo (npm workspaces)polyrepoOne contract source; atomic cross-cutting changes; dependency-light.
REST /v1 + NestJSGraphQL / tRPCMaps cleanly to the resource model and to PDF/JSON/CSV exports.
RLS + app-layer scope (both)app-layer onlyDefense in depth; RLS is the backstop if a query escapes the chokepoint.
Transactional outboxdirect dual-writeOnly correct way to keep DB + search/email/webhooks consistent.
Scanner-adapter interfacehard-coded scannersPluggable, testable, degrades offline, safe-by-construction.
Deterministic fallback plannerLLM-onlyGuarantees a functional pentest workflow with no external AI in the sandbox.
No live exploitationpayload firingLegal/ethical requirement; validation modeled as gated state.

§12 Threat Model

Five trust boundaries, each with its own controls; top risks R1–R8 each map to an invariant and a mitigation.

1 · Browser ↔ API

JWT cookie · CSRF/Origin guard · Zod validation.

2 · API ↔ Postgres

RLS · parameterized queries · least-privilege role.

3 · Worker ↔ target network

SSRF guard · authorization scope · isolated egress.

4 · API ↔ LLM / target text

Untrusted-data handling · prompt-injection mitigation.

5 · API ↔ third-party webhooks

HMAC verify-before-DB. Top risks R1–R8 — authorization scope, SSRF, secret handling, cross-tenant leakage, audit tampering, scan-proxy abuse, prompt injection, artifact handling — each map to an invariant (§9) and a mitigation (§5).

← Gallery