Proof · Technical Reference
Deployment reference for EthiCompass Proof. Written for the people who have to run it. On-premise and private-cloud configurations for regulated enterprises. Structured sizing for every scenario, from minimum viable deployment to high-concurrency production.
This page is a technical reference. For commercial discussion, implementation timelines, and engagement scope, contact our team.
Deployment Modes
Proof ships as Docker containers and supports two deployment modes. Both preserve the registered, hash-citable audit trail, the 8-dimension evaluation framework and the EU AI Act Articles 9–15 mapping. What differs is whose infrastructure the workload runs on. What does not differ is the boundary: under both, your agent’s output stays inside it. Each mode states that plainly below.
Deployed inside your data center. All inference runs locally on dedicated GPUs. Suitable for the strictest data residency and regulatory requirements, including sectors with absolute on-premise mandates.
What leaves your networkNothing leaves your network. The installation resolves everything internally and needs no outbound access — your network team opens no firewall rule for it to work. The models that attack and the model that judges are open-weight and run inside the installation, so no turn of your agent is sent anywhere to be generated or scored.
Deployed in your AWS, Azure, or GCP tenancy. Inference runs on managed GPU instances within your network boundary. Suitable for organisations with approved cloud regions and residency controls.
What leaves your networkNothing leaves your cloud account. The same containers run on managed GPU instances inside your tenancy, and the models still run in the installation. The boundary moves from your data centre to your cloud account; it does not widen.
Deployment mode is a decision made during the Architecture Assessment phase of implementation. All modes produce the same compliance evidence and the same registered, hash-citable audit trail.
Reference Architecture
Proof is composed of infrastructure services and evaluation models. The infrastructure services manage data storage, versioning, messaging, and the application layer. The evaluation models produce the quantitative scores behind the 8-dimension framework.
| Component | Role |
|---|---|
| MinIO | S3-compatible object storage for datasets and evidence artifacts |
| LakeFS | Versioned dataset management with full lineage |
| Valkey | Ephemeral work coordination — atomic claim and heartbeat, with TTL. No persistence by design: losing coordination does not lose evidence, which lives in object storage. |
| API Orchestrator (FastAPI) | Coordinates evaluation requests across dimensions |
| Metrics Service (FastAPI) | Aggregates scores and produces the Score Card |
| Model | Role in the 8-Dimension Framework | Served Via |
|---|---|---|
| Qwen3-Embedding (600M parameters) | Embeddings for retrieval-grounded scoring across the dimensions | vLLM (embedding task) |
| Gemma 4 E4B (4.7B parameters) | Independent judge (LLM-as-Judge) for dimension scoring | vLLM |
| Llama Guard 4 (12B parameters) | Guardian model for Safety & Harmful Content detection | vLLM |
All models are served through vLLM with paged attention, continuous batching, and efficient KV cache management. The embedding model is shared across two dimensions and runs as a single instance.
What this list does not coverA production run uses six model roles, and the three above are the embedder, the judge and the guardian. The generator, the attackers and the reranker also run inside your installation and are not sized on this page. Treat the figures below as the floor for three of six roles, not as the requirement for a deployment — and ask us for the rest before you buy hardware.
Configuration Profiles
Three reference configurations. Each is validated for a specific deployment posture. Select the profile that matches your evaluation concurrency, residency requirements, and regulatory commitments.
| Resource | MinimumINT4, sequential | RecommendedINT4, parallel | Productionbf16, high concurrency |
|---|---|---|---|
| GPU | 1 × 24 GB VRAM (e.g., RTX 4090, A5000) | 1 × 24 GB VRAM (e.g., RTX 4090, A5000) | 1 × 80 GB VRAM (A100) or 2 × 48 GB (A6000) |
| System RAM | 32 GB DDR4/DDR5 | 64 GB DDR5 | 128 GB DDR5 ECC |
| CPU | 14 cores | 20 cores | 32 cores |
| Storage | 256 GB NVMe SSD | 512 GB NVMe SSD | 1 TB NVMe SSD |
| Network | 1 Gbps | 1 Gbps | 10 Gbps |
| Use case | Proof of concept, low-volume evaluation | Standard enterprise deployment | Multi-tenant, high-concurrency, real-time monitoring |
All profiles assume a single-node deployment. Multi-node clustering is available for production environments with failover requirements. Discuss during Architecture Assessment.
GPU & Inference Sizing
These three models — the embedder, the judge and the guardian — have materially different memory profiles. They are three of the six roles a production run uses; the generator, the attackers and the reranker are not sized here. The deployment can run in full precision, INT8, or INT4. Each scenario is a tradeoff between VRAM footprint, throughput and evaluation latency.
On quantization and the scoreWe do not claim that a quantized judge produces the same score as one at full precision. The score is the deliverable, so drift under quantization is a question about the validity of the evidence, not a performance footnote. Ask us what we have measured per dimension before you treat a quantized profile as audit-grade.
| Scenario | Precision | Qwen3 (0.6B) | Gemma 4 (4.7B) | Llama Guard 4 (12B) | Total VRAM |
|---|---|---|---|---|---|
| A — Full precision | bf16 | ~2.5 GB | ~12.0 GB | ~27.7 GB | ~42.2 GB |
| B — INT4 quantized | embeddings bf16, models INT4 | ~2.5 GB | ~5.0 GB | ~9.7 GB | ~17.2 GB |
| C — INT8 quantized | embeddings bf16, models INT8 | ~2.5 GB | ~7.0 GB | ~15.7 GB | ~25.2 GB |
| Scenario | Single-GPU Option | Multi-GPU Option |
|---|---|---|
| A (bf16) | 1 × NVIDIA A100 80 GB | 2 × NVIDIA A6000 48 GB |
| B (INT4) | 1 × NVIDIA RTX 4090 24 GB | 1 × NVIDIA A5000 24 GB |
| C (INT8) | 1 × NVIDIA L40 48 GB | 1 × NVIDIA A6000 48 GB |
All sizing above assumes single-request baseline per model. vLLM handles multiple concurrent requests through continuous batching, which grows the KV cache pool proportionally.
If evaluations run sequentially (one model active at a time), the inactive model can be offloaded from GPU memory. The platform then only requires the largest model plus the embeddings:
Sequential execution reduces VRAM but increases end-to-end evaluation latency. Suitable for audit-style evaluation or low-volume pilots. Not recommended for real-time monitoring of production AI systems.
Infrastructure Services
The infrastructure services run on CPU and system RAM. They are independent of the GPU workload and can be co-located on the same node as the evaluation models or distributed across dedicated hosts.
| Component | RAM (min) | RAM (rec) | CPU (min) | CPU (rec) | Storage |
|---|---|---|---|---|---|
| MinIO | 2 GB | 4 GB | 2 cores | 4 cores | 50 GB NVMe SSD min, variable |
| LakeFS | 1 GB | 2 GB | 1 core | 2 cores | 10 GB for metadata |
| LakeFS PostgreSQL | 1 GB | — | — | — | Included with LakeFS |
| Valkey | 1 GB | 2 GB | 1 core | 2 cores | None — ephemeral by design |
| API Orchestrator (FastAPI) | 256 MB | 512 MB | 1 core | 1 core | — |
| Metrics Service (FastAPI) | 512 MB | 1 GB | 2 cores | 2 cores | — |
MinIO is I/O-intensive. NVMe SSD is required for any deployment handling datasets larger than 10 GB. Direct-attached storage is preferred over network-attached storage for the MinIO volume.
Storage, Network & Operating System
| Item | Allocation |
|---|---|
| Operating system and Docker runtime | 30 GB |
| Docker images (services) | 15 GB |
| Evaluation model weights (from Hugging Face) | ~35 GB |
| MinIO dataset storage | 50 GB minimum, variable with use |
| Valkey | none — ephemeral |
| LakeFS metadata | 10 GB |
| Total storage | 160 GB minimum, NVMe SSD |
Next Steps
This page is a reference, not a procurement checklist. Every deployment is calibrated to the organization's regulatory posture, AI system inventory, and existing infrastructure. The Architecture Assessment — part of a standard implementation — turns this reference into a concrete deployment plan.
A focused technical session with our platform team. We review your environment, your residency constraints, and your expected evaluation volume. You leave with a sized deployment plan.
If you are earlier in evaluation, our team can walk through the product, the 8-dimension framework, and deployment options in a 30-minute conversation.
For the full product overview, differentiators, and implementation timeline, return to the Proof product page.
Back to ProofLast updated: April 2026