Sovereign On-Premise Agentic AI Workbench using Open-Weight Multimodal LLMs for Confidential Industrial Work

Sovereign On-Premise Agentic AI Workbench using Open-Weight Multimodal LLMs for Confidential Industrial Work

1. Executive Snapshot

Summary

Mangalore Refinery and Petrochemicals Limited (MRPL) and other strategic public sector undertakings (PSUs) require an air-gapped, self-hosted agentic AI workbench to automate routine but highly sensitive knowledge work—including engineering drawing reviews (P&IDs), scanned inspection reports, approval notes, financial calculations, and internal tool development [PS]. Cloud-based LLM services (e.g., ChatGPT, Claude) are strictly prohibited due to severe data confidentiality risks regarding unreleased designs, vendor negotiations, and plant safety schematics [PS]. The proposed solution, SovereignBench, is a fully offline, multi-model agentic AI platform deployed on local enterprise GPU hardware [INFERENCE]. It dynamically routes tasks to specialized open-weight models (e.g., vision-language models for scanned drawings/notes, coding models for sandboxed script execution, and reasoning models for document synthesis) [INFERENCE]. Operating through a deterministic tool-calling agent framework (LangGraph), the platform reads local documents, searches internal SOPs via a local vector database, executes code inside isolated containers, generates native Microsoft Office deliverables (.docx, .xlsx, .pptx), and provides live GUI-integrated network monitoring to prove zero outbound network calls [INFERENCE].

Metadata Table

Fundamental System Objective

To deliver a 100% air-gapped, multi-model agentic workbench running on local GPU hardware that automatically routes multimodal industrial tasks to specialized open-weight LLMs, executes multi-step workflows with local tool execution and local RAG grounding, generates real office file deliverables, and continuously proves network isolation via real-time packet inspection [PS].

2. Problem Deconstruction

The explicit problem statement provided by MRPL has been extracted into atomic requirements below, retaining all explicit conditions and deriving necessary technical implications.

Explicit and Inferred Requirements

1. Requirement 1 (On-Premise & Air-Gapped Execution)

   • Source Text: "A self-hosted, air gapped AI workbench running entirely on the organization's own GPU server. Nothing leaves the premises." [PS]
   • Interpretation: The system must run completely offline without relying on external cloud APIs, external license verification servers, or remote telemetry [INFERENCE].
   • Implementation Implication: Model weights, vector stores, OCR runtimes, frontend static assets, and Python dependencies must be pre-packaged locally in Docker containers or python virtual environments [INFERENCE]. Network interfaces must be monitored or isolated via local firewall rules [ASSUMPTION].
   • Confidence Tag: [PS]

2. Requirement 2 (Multi-Model Architecture & Dynamic Model Auto-Selection)

   • Source Text: "The backend should not be locked to one model. It needs to support multiple open weight models at once and automatically pick the right one for a given task based on what that task needs, a coding request handled differently from a document summary request." [PS]
   • Interpretation: A router component must analyze user intent and input data types (code, text, image, tabular data) and dispatch the request to the optimal locally hosted open-weight model [INFERENCE].
   • Implementation Implication: Local model serving must expose standardized OpenAI-compatible APIs (e.g., via Ollama or vLLM) hosting distinct model endpoints (e.g., Qwen2.5-Coder for coding, Qwen2.5-VL for images/drawings, DeepSeek-R1-Distill/Llama-3.1 for complex reasoning) [RESEARCHED].
   • Confidence Tag: [PS]

3. Requirement 3 (Extensible Model Registry)

   • Source Text: "New open weight models should be addable later without redesigning the system, since this space is moving fast." [PS]
   • Interpretation: The backend router and inference engine must decouple model orchestration from specific model identifiers via a configurable model registry abstraction [INFERENCE].
   • Implementation Implication: A JSON/YAML configuration file or admin UI allows administrators to register new local GGUF/vLLM endpoints, specify capabilities (vision, coding, context length), and update routing rules without code modification [INFERENCE].
   • Confidence Tag: [PS]

4. Requirement 4 (Agentic Loop & Local Tool Execution)

   • Source Text: "The assistant also needs to actually act like an agent. Plan out multi step work, call local tools such as file read and write, code execution in a sandbox, spreadsheet work, internal document search, and iterate on a task instead of answering once and stopping." [PS]
   • Interpretation: The system must implement a ReAct/state-graph loop where the model formulates plans, executes tools deterministically, inspects intermediate tool outputs, self-corrects errors, and continues until completion [INFERENCE].
   • Implementation Implication: Use an agent framework (LangGraph) integrated with sandboxed execution environments (Docker/Pyodide container), local file system adapters, and spreadsheet libraries (openpyxl) [RESEARCHED].
   • Confidence Tag: [PS]

5. Requirement 5 (Multimodal Processing)

   • Source Text: "It needs to handle more than text too: scanned PDFs, handwritten notes, engineering drawings, photographs, read through on device OCR and vision models." [PS]
   • Interpretation: Documents uploaded by users may lack digital text layers (scanned paper, handwritten maintenance logs, P&ID schematics) [INFERENCE].
   • Implementation Implication: Ingestion pipeline must integrate on-device OCR engines (Docling / PaddleOCR) and vision-language models (Qwen2.5-VL) to extract structured text, spatial bounding boxes, and symbol interpretations without cloud OCR [RESEARCHED].
   • Confidence Tag: [PS]

6. Requirement 6 (Real Deliverables Output Generation)

   • Source Text: "Output should be real deliverables, approval notes, PPT/Word/Excel files, working code, calculations with steps shown, not just chat replies." [PS]
   • Interpretation: Users expect structured, downloadable industrial artifacts, not merely inline chat text [INFERENCE].
   • Implementation Implication: Implement code execution and document-generation tools (python-docx, openpyxl, python-pptx) that convert agent outputs into standard file formats, with explicit mathematical verification steps rendered in documents [RESEARCHED].
   • Confidence Tag: [PS]

7. Requirement 7 (Local Knowledge Base Grounding - RAG)

   • Source Text: "ground itself in the organization's own manuals, SOPs and past correspondence through a local knowledge base connector, again with nothing going external." [PS]
   • Interpretation: The agent must pull contextual information from internal repositories via semantic RAG without sending data out [INFERENCE].
   • Implementation Implication: Deploy an air-gapped vector database (Qdrant or pgvector) with local embedding models (e.g., bge-large-en-v1.5 or nomic-embed-text) to chunk, index, and retrieve internal SOPs and manuals [RESEARCHED].
   • Confidence Tag: [PS]

8. Requirement 8 (Hardware Adaptability & Demonstration Proof)

   • Source Text: "A working local deployment, demonstrable on a single workstation or server with a mid range GPU... shows model auto selection across at least two different task types... coding task run and verified in a sandbox... multimodal task... proof of air-gap through logs or a visible network monitor that no external calls are made." [PS]
   • Interpretation: The hackathon MVP must run smoothly on a standard single GPU workstation (e.g., RTX 4090 / 24GB VRAM) using quantized models (Q4_K_M/Q8_0), with built-in network capture tools to physically demonstrate air-gap security [INFERENCE].
   • Implementation Implication: Include an embedded real-time network traffic widget monitoring port activity and interface traffic via tcpdump/iptables logs [ASSUMPTION].
   • Confidence Tag: [PS]

3. Stakeholders & User Needs

4. Requirements Traceability

5. Constraints & Environment Realities

Operational Environment Constraints

1. Air-Gap Strictness: PSU enterprise networks at refinery locations (e.g., MRPL Mangalore) strictly disable outbound internet traffic on industrial networks [KNOWN]. The workbench must function cleanly with network interfaces physically disconnected or blocked by enterprise firewalls [PS].
2. GPU Hardware Budget: While production deployment may utilize multi-A100/H100 clusters, hackathon evaluation occurs on workstations with mid-range consumer or workstation GPUs (e.g., single NVIDIA RTX 4090 24GB or RTX A6000 48GB) [PS]. Total VRAM constraints require model quantization (4-bit GGUF or AWQ) and offloading strategies [RESEARCHED].
3. Power and Thermal Resilience: Industrial GPU servers operate in climate-controlled server rooms, but hackathon evaluation hardware may face thermal throttling under continuous multi-model inference [INFERENCE].
4. Data Formats in Refining: Documents include legacy scanned PDFs from the 1990s, skewed P&IDs (A0/A1 blue-prints), multi-page Excel sheets with complex macro formulas, and handwritten maintenance logs in varying handwriting styles [KNOWN].
5. Language & Terminology: Documents contain specialized petrochemical engineering terminology (e.g., FCCU, hydrocracker, ASME standards, ISA-5.1 P&ID symbols) requiring domain-accurate grounding [KNOWN].

6. Research Questions

1. How can dynamic model auto-selection be executed with sub-second latency on a single GPU without constantly swapping model weights in VRAM? [RESEARCHED]
2. Which open-weight vision-language models perform best on dense technical schematics (P&IDs) and low-resolution scanned engineering reports under 15B parameters? [RESEARCHED]
3. What is the most secure and reliable architecture for sandbox code execution within a Python backend operating in an air-gapped environment? [RESEARCHED]
4. How can generated documents (.docx, .xlsx, .pptx) adhere strictly to organizational corporate formatting without requiring cloud templates? [INFERENCE]
5. What specific Linux network tracing mechanism (iptables, eBPF, tcpdump) provides undeniable, visual proof of zero outbound traffic during live hackathon judging? [RESEARCHED]

7. Data Requirements & Sources

Data Classification Categories

• A. Explicitly Mentioned Data: Scanned PDFs, handwritten notes, engineering drawings (P&IDs), financial documents, vendor negotiations, SOPs, past correspondence [PS].
• B. Inferred Data Requirements: Standardized document templates (.docx approval note structures, .xlsx calculations templates) [INFERENCE].
• C. Collected / Synthetic Data: Synthesized MRPL-like inspection report PDFs and open-source P&ID samples for demo purposes [ASSUMPTION].
• D. Data Requiring Validation: Exact VRAM footprint of concurrent multi-model serving under batch size 1 [NEEDS VALIDATION].

8. Existing Solutions & Gap Analysis

9. Candidate Solution Approaches

+---------------------------------------------------------------------------------------------------+
|                                  CANDIDATE ARCHITECTURE COMPARISON                                |
+--------------------------------------------------+------------------------------------------------+
| APPROACH 1: Monolithic Single-Model Script       | APPROACH 2: Dynamic Multi-Model Agent (CHOSEN) |
| - Uses 1 general model for all tasks             | - Router dispatches task to specialized LLM    |
| - Simple linear script pipeline                  | - LangGraph state graph with ReAct loops       |
| - Low flexibility; high VRAM bloat               | - Docker isolated sandbox + Native Office tool |
| - Fails specialized coding & dense vision        | - Optimal performance on mid-range GPU         |
+--------------------------------------------------+------------------------------------------------+

Approach 1: Monolithic Single-Model Pipeline with Script-Based Orchestration

• Mechanism: Single large model (e.g., Llama-3.1-70B or Qwen2.5-72B quantized) loaded into VRAM, handled by a fixed Python sequential script.
• Accuracy Potential: Medium-Low. A single generalist quantized model underperforms compared to specialized models (e.g., Qwen2.5-Coder for code, Qwen2.5-VL for P&IDs) [RESEARCHED].
• Complexity: Low implementation complexity.
• Resource Overhead: Extremely high VRAM requirements (requires >40GB VRAM, impossible on single consumer GPU) [RESEARCHED].
• Hackathon Feasibility: Poor due to hardware constraints on venue workstations [INFERENCE].

Approach 2: Dual-Model Dynamic Routing Agentic Workbench with Containerized Sandbox (RECOMMENDED)

• Mechanism: A lightweight intent classifier routes tasks to specialized quantized models hosted concurrently on Ollama/vLLM. Coding goes to Qwen2.5-Coder-14B, multimodal P&ID/scanned PDF vision goes to Qwen2.5-VL-7B, and general orchestration/reasoning uses DeepSeek-R1-Distill-Qwen-14B [RESEARCHED]. A LangGraph state graph orchestrates ReAct tool loops, invoking local tools (Docling OCR, Pyodide/Docker execution, python-docx file generation) [RESEARCHED].
• Accuracy Potential: Extremely High. Specialized open-weight models match or exceed general models in their respective domains [RESEARCHED].
• Complexity: Moderate. Decoupled microservices architecture.
• Resource Overhead: Optimized VRAM (~18–22 GB total VRAM), allowing execution on a single 24GB GPU (RTX 4090/A5000) [RESEARCHED].
• Hackathon Feasibility: Excellent. Demonstrates true dynamic routing, multimodal intelligence, and full air-gap security within standard hackathon GPU budgets [INFERENCE].

Approach 3: Cloud-Fallback Hybrid Architecture

• Mechanism: Tries local LLM first; falls back to cloud API (e.g., OpenAI/Anthropic) if local task fails.
• Rejection Reason: STRICTLY REJECTED. Violates the fundamental core mandate of MRPL [PS]. Zero bytes are permitted to leave the premises [PS].

10. Recommended Approach & Why

Selected Architecture: Approach 2 (Dynamic Multi-Model Agentic Workbench)

Technical Justification

1. Fits Hardware Realities: By deploying quantized specialized models (e.g., Qwen2.5-Coder-14B-Instruct-Q4_K_M ~9GB VRAM + Qwen2.5-VL-7B-Instruct-Q4_K_M ~5GB VRAM + bge-large-en-v1.5 ~1.3GB VRAM), total VRAM footprint remains under 18GB [RESEARCHED]. This enables smooth performance on a single 24GB GPU without VRAM thrashing [RESEARCHED].
2. Superior Domain Performance: Specialized models beat generalist models of larger sizes in specific domain benchmarks (Qwen2.5-Coder-32B/14B matches top propriety coders; Qwen2.5-VL excels at document structure and spatial graphic understanding) [RESEARCHED].
3. True Agentic ReAct Execution: Using LangGraph, the agent can autonomously generate code, test it in the sandbox, read error tracebacks, fix its code, and output verified results without stopping at an initial incorrect response [RESEARCHED].
4. Defensible Air-Gap Compliance: The decoupled local container architecture makes it simple to bind all services to local loopback interfaces (127.0.0.1 / Docker internal bridge) and display network interface activity in real time [INFERENCE].

11. Technical Architecture

+---------------------------------------------------------------------------------------------------+
|                                     SOVEREIGNBENCH ARCHITECTURE                                   |
+---------------------------------------------------------------------------------------------------+
|                                  USER INTERFACE LAYER (React + Tailwind)                          |
|  - Workspace UI / Chat Canvas  - Deliverable File Viewer  - Live Network Monitor Dashboard Widget |
+---------------------------------------------------------------------------------------------------+
                                                  |  (REST / WebSocket - Localhost)
+---------------------------------------------------------------------------------------------------+
|                                   BACKEND ORCHESTRATION (FastAPI)                                 |
|  - Request Ingestion & Sanitization               - Model Registry & Intent Router Engine         |
|  - LangGraph State Machine & Agentic Loop         - Audit Logger & Compliance Ledger              |
+---------------------------------------------------------------------------------------------------+
            |                                     |                                    |
            v                                     v                                    v
+-----------------------+   +------------------------------------+   +------------------------------+
| INFERENCE ENGINE      |   | AGENT TOOL EXECUTION ENGINE        |   | LOCAL DATA & RAG ENGINE      |
| (Ollama / vLLM API)   |   |                                    |   |                              |
| - Qwen2.5-Coder-14B   |   | - Docling / PaddleOCR Engine       |   | - Qdrant Vector Database     |
| - Qwen2.5-VL-7B       |   | - Pyodide/Docker Sandbox Runtime   |   | - Local BGE Embedding Engine |
| - DeepSeek-R1-Distill |   | - Office Generators (.docx/.xlsx)  |   | - SQLite Document Store      |
+-----------------------+   +------------------------------------+   +------------------------------+
                                                  |
+---------------------------------------------------------------------------------------------------+
|                                HARDWARE & AIR-GAP FIREWALL LAYER                                  |
|  - Single Server / Workstation (NVIDIA GPU 24GB VRAM)                                             |
|  - Air-Gapped Network Configuration (iptables drop all egress / eBPF network packet filter)       |
+---------------------------------------------------------------------------------------------------+

Detailed Data-Flow Narrative

1. Task Submission: A process engineer uploads a scanned PDF inspection report with an engineering drawing and enters a prompt: "Extract findings, calculate pipe corrosion rate, and generate an executive approval note (.docx)."
2. Intent Classification & Model Selection: FastAPI backend receives the request. The Intent Router inspects the payload. Detecting an image/scanned PDF upload and a document generation task, it selects Qwen2.5-VL-7B for vision extraction and DeepSeek-R1-Distill-14B for reasoning and drafting [INFERENCE].
3. Multimodal Ingestion: The file is routed to the local Docling/PaddleOCR engine. Layout parsing extracts image regions, tables, and handwritten text layers locally [RESEARCHED].
4. Agent Planning & Knowledge Retrieval: The LangGraph ReAct agent initializes state. It queries the local Qdrant vector database using bge-large-en-v1.5 to retrieve MRPL corrosion allowance standards from internal SOPs [RESEARCHED].
5. Tool Call - Sandboxed Math Calculation: The agent formulates a Python script to compute corrosion rates based on extracted inspection numbers. It passes the script to the Docker/Pyodide Sandbox Tool. The script runs locally, verifies mathematical outputs, and returns structured JSON [INFERENCE].
6. Tool Call - Office Deliverable Generation: The agent passes structured findings to the python-docx Tool, applying the official MRPL approval note template. The file Approval_Note_Corrosion_Report.docx is written to local disk (/data/deliverables/) [INFERENCE].
7. Verification & UI Rendering: The backend returns the final response, displaying execution steps, intermediate calculations, inline previews, and a direct download link for the generated .docx file [INFERENCE].
8. Continuous Air-Gap Audit: Throughout the transaction, the eBPF/iptables network monitor logs network traffic on all network cards, rendering a live graph showing 0 outbound bytes to external IPs [INFERENCE].

12. Technology Stack

13. AI/ML Justification

AI Component Evaluation Matrix

Where AI Should NOT Be Used

• Calculations: Mathematical formula execution (corrosion rates, pressure drop, financial totals) must NOT rely on LLM arithmetic generation. The LLM must write a Python script, which is executed deterministically in the sandbox to guarantee 100% mathematical precision [INFERENCE].
• File Formatting: Layout formatting for .docx, .xlsx, and .pptx files must NOT be generated as raw raw text. Standardized Python libraries (python-docx, openpyxl) must deterministically build documents from JSON schema outputs [INFERENCE].
• Network Security Verification: Network monitoring must NOT involve AI prediction. It uses raw Linux kernel packet counters (iptables / eBPF) [INFERENCE].

14. Data Architecture & Offline/Low-Network Design

+---------------------------------------------------------------------------------------------------+
|                                 DATA ARCHITECTURE & LOCAL STORAGE                                 |
+---------------------------------------------------------------------------------------------------+
|                                                                                                   |
|  LOCAL FILE STORAGE (/data/workspace)                                                             |
|  +-- /uploads/         --> User-uploaded scanned PDFs, P&IDs, Excel sheets                         |
|  +-- /models/          --> GGUF / AWQ quantized model weights (Qwen, DeepSeek, BGE)               |
|  +-- /deliverables/    --> Generated .docx, .xlsx, .pptx files                                    |
|                                                                                                   |
|  EMBEDDED VECTOR DB (Qdrant Local Docker)                                                         |
|  +-- Collection: "refinery_sops"      (Dense vectors: 1024-dim, BGE-Large)                         |
|  +-- Collection: "correspondence"     (Metadata filtering: department, date, classification)      |
|                                                                                                   |
|  LOCAL RELATIONAL DB (SQLite / PostgreSQL Local)                                                 |
|  +-- Tables: users, agent_threads, tool_execution_logs, system_audit_trail                        |
|                                                                                                   |
+---------------------------------------------------------------------------------------------------+

Local Storage Schema (SQLite - Core Tables)

-- Thread state for LangGraph multi-step loops
CREATE TABLE agent_threads (
    thread_id TEXT PRIMARY KEY,
    user_id TEXT NOT NULL,
    task_type TEXT NOT NULL, -- 'coding', 'multimodal', 'document_summary'
    selected_model TEXT NOT NULL,
    current_status TEXT NOT NULL, -- 'planning', 'executing_tool', 'completed', 'failed'
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Execution audit log for security & CISO validation
CREATE TABLE tool_execution_logs (
    log_id INTEGER PRIMARY KEY AUTOINCREMENT,
    thread_id TEXT NOT NULL,
    tool_name TEXT NOT NULL, -- 'sandbox_python', 'docling_ocr', 'docx_builder'
    input_payload TEXT,
    output_payload TEXT,
    execution_time_ms INTEGER,
    status TEXT NOT NULL,
    FOREIGN KEY(thread_id) REFERENCES agent_threads(thread_id)
);

Zero External Network & Offline UX Strategies

1. Zero Outbound Sockets: All microservices (FastAPI, Qdrant, Ollama, React static server) listen exclusively on 127.0.0.1 or internal Docker bridge networks (172.18.0.0/16). No external network interfaces are exposed [INFERENCE].
2. Local Static Asset Bundling: React frontend, fonts (Inter/Roboto), icons (Lucide-React), and JS packages are bundled statically into the Docker image. Zero CDN calls (e.g., Google Fonts, unpkg, Tailwind CDN) are made [INFERENCE].
3. Local Weight Pre-fetching: All LLM weights, vision projectors, and embedding checkpoints are loaded from local host directories (/models/), preventing HuggingFace runtime check calls [INFERENCE].
4. Resilient Local Queueing: If a heavy vision job is processing, backend background tasks manage GPU memory locks to prevent VRAM out-of-memory crash loops [INFERENCE].

15. Security, Privacy & Governance

Air-Gap Enforcement & Network Firewall

• Kernel Level Rules: The deployment script configures iptables rules blocking all outbound TCP/UDP traffic except local loopback: iptables -A OUTPUT -o lo -j ACCEPT iptables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT iptables -P OUTPUT DROP [RESEARCHED].
• Live Network Monitor Widget: The frontend embeds a WebSocket-based packet auditor querying /api/v1/network/stats. It displays real-time bandwidth (0 KB/s outbound), open ports, and dropped outbound packet counters to convince hackathon judges [INFERENCE].

Sandboxing & Code Execution Security

• Docker Container Isolation: Code generated by the coding LLM is executed inside an ephemeral Docker container running under a restricted non-root user (nobody), with --net=none (zero network access), --memory=512m, --cpus=1.0, and read-only filesystem mounts (except /tmp) [RESEARCHED].
• Process Timeout Limits: Sandbox execution is terminated after 10 seconds to prevent infinite loops or memory leaks [INFERENCE].

Access Control & Audit Governance

• Role-Based Access Control (RBAC): Users log in via local OAuth2 / JWT session handling. Roles include Engineer (can run workflows, view deliverables), SOP Admin (can upload new manuals to vector store), and CISO Auditor (can view execution logs and network audit trails) [INFERENCE].
• Immutable Execution Audit Logs: Every LLM prompt, tool call, code execution stdout/stderr, and file generated is logged to a local SQLite audit table with SHA-256 hash checks to prevent tampering [INFERENCE].

16. MVP Scope (Hackathon Build)

+---------------------------------------------------------------------------------------------------+
|                                      HACKATHON MVP BUILD SCOPE                                    |
+---------------------------------------------------------------------------------------------------+
|  [MUST BUILD]                                                                                     |
|  - Air-gapped Docker deployment bundle with zero egress configuration.                            |
|  - Dynamic intent classifier routing across 2+ models (Qwen2.5-Coder + Qwen2.5-VL).              |
|  - LangGraph agent loop performing multi-step planning & reflection.                             |
|  - Docling/PaddleOCR pipeline parsing scanned PDFs and P&ID diagrams.                              |
|  - Python code execution sandbox returning verified calculations.                                |
|  - Native Word (.docx) & Excel (.xlsx) deliverable generator.                                     |
|  - Real-time Network Traffic Audit UI widget (showing 0 outbound packets).                        |
|                                                                                                   |
|  [SHOULD BUILD]                                                                                   |
|  - Qdrant RAG index pre-loaded with sample MRPL refinery SOPs.                                    |
|  - Powerpoint (.pptx) board presentation deliverable tool.                                        |
|  - UI Model Registry manager (view active local models & VRAM usage).                             |
|                                                                                                   |
|  [DESCRIBED ONLY / OUT OF SCOPE FOR HACKATHON]                                                    |
|  - Enterprise Active Directory / LDAP SSO integration.                                            |
|  - Multi-node multi-GPU Distributed vLLM Cluster orchestration.                                   |
|  - Direct integration with physical SCADA / Industrial IoT DCS networks.                          |
+---------------------------------------------------------------------------------------------------+

17. Live Demo Simulation Plan

To deliver an undeniable demo during SIH judging on a single mid-range GPU workstation without relying on external services:

1. Simulated Refinery Scenario 1: Multimodal P&ID & Scanned Inspection to Approval Note
   • Action: Upload a sample scanned equipment inspection report PDF containing a skewed P&ID schematic section and handwritten corrosion measurements.
   • Demonstration: Show Qwen2.5-VL extracting visual bounding boxes, reading handwritten numbers, and passing them to the math sandbox tool. Show the agent calling python-docx to generate Corrosion_Inspection_Approval_Note.docx. Download and open the generated Word file live [INFERENCE].
2. Simulated Refinery Scenario 2: Sandboxed Tool Coding & Data Processing
   • Action: Prompt the workbench: "Parse this raw CSV of plant temperature logs, compute 24-hour moving averages, identify anomalous spikes, and generate an Excel summary sheet with formatted headers."
   • Demonstration: Show the router selecting Qwen2.5-Coder-14B. Watch the agent write Python code, execute it inside the sandbox, capture the output, auto-correct a syntax warning, and produce Plant_Temperature_Analysis.xlsx [INFERENCE].
3. Simulated Refinery Scenario 3: Grounded RAG Querying
   • Action: Ask a question regarding MRPL emergency shutdown protocols for Hydrocracker Unit 2.
   • Demonstration: Show local vector search in Qdrant pulling relevant sections from pre-indexed OSHA/refinery SOPs and displaying paragraph-level source citations [INFERENCE].
4. The Ultimate Sovereign Proof: Live Network Disconnection & Packet Monitor
   • Action: Open the live Network Monitor pane on the UI. Run tcpdump or trigger outbound requests to google.com or api.openai.com from a terminal to show they time out or get blocked by iptables.
   • Demonstration: Show that the workbench completes the entire agentic multi-step workflow with 0 outbound bytes, proving 100% air-gapped sovereignty [INFERENCE].

18. Implementation Roadmap

Project Timeline Target: 30 September 2026

+---------------------------------------------------------------------------------------------------+
|                                      IMPLEMENTATION ROADMAP                                       |
+---------------------------------------------------------------------------------------------------+
|  PHASE 1: Core Setup & Air-Gap Environment (Weeks 1-2)                                            |
|  - Setup local GPU workstation (CUDA, vLLM/Ollama, Docker engine).                                |
|  - Download and quantize Qwen2.5-Coder-14B & Qwen2.5-VL-7B GGUF weights.                          |
|  - Configure local iptables air-gap firewall scripts & local network monitoring API.             |
|                                                                                                   |
|  PHASE 2: Agent Engine & Local Tools (Weeks 3-4)                                                  |
|  - Implement LangGraph state machine with ReAct loop.                                            |
|  - Build Docling/PaddleOCR document parsing microservice.                                         |
|  - Build Sandboxed Python container execution tool.                                               |
|  - Build python-docx, openpyxl, and python-pptx deliverable generator tools.                      |
|                                                                                                   |
|  PHASE 3: Dynamic Router & RAG Engine (Weeks 5-6)                                                 |
|  - Develop lightweight Intent Classifier & Model Registry manager.                                 |
|  - Deploy Qdrant local Docker instance & load BGE embedding pipeline.                             |
|  - Index sample industrial SOPs and synthetic refinery inspection reports.                       |
|                                                                                                   |
|  PHASE 4: UI Development & Deliverable Viewer (Weeks 7-8)                                         |
|  - Build React + Tailwind frontend with multi-pane workspace layout.                              |
|  - Implement live agent execution timeline, thought trace viewer, and network monitor widget.     |
|  - Integrate document previewers (.docx/.xlsx download handlers).                                 |
|                                                                                                   |
|  PHASE 5: End-to-End Testing & SIH Demo Optimization (Weeks 9-10)                                 |
|  - Run stress tests on single 24GB GPU (VRAM management, latency checks).                         |
|  - Execute dry-run demos across all 3 primary evaluation scenarios.                               |
|  - Finalize offline installer scripts & self-contained Docker compose bundle.                     |
+---------------------------------------------------------------------------------------------------+

19. Testing & Validation Plan

20. Risk Register

21. Cost & Scalability

Hackathon Prototype (Single GPU Workstation Setup)

• Compute Hardware: Existing student/lab workstation with 1x NVIDIA RTX 4090 (24GB VRAM) or RTX A5000 (24GB VRAM) – ₹0 (Utilizing available hackathon/university hardware) [INFERENCE].
• Software Costs: 100% Open Source (React, FastAPI, LangGraph, Ollama, Qwen2.5, Qdrant, Docker, Linux) – ₹0 [KNOWN].
• Total Hackathon Cost: ₹0 [INFERENCE].

Production Enterprise Deployment (MRPL On-Premise Data Center)

• Server Hardware: 1x Enterprise 2U GPU Server (e.g., Dell PowerEdge / HP ProLiant with 2x NVIDIA L40S 48GB GPUs or 2x H100 80GB GPUs) – [NEEDS VALIDATION] (Estimated ₹15,000,000 – ₹25,000,000 enterprise CAPEX) [INFERENCE].
• High Availability & Storage: On-premise NVMe SAN array for document vectors and model caching – [NEEDS VALIDATION].
• Operational Costs: Electricity, cooling, and internal IT infrastructure maintenance – Standard PSU datacenter OPEX [INFERENCE].
• Software License Fees: ₹0 (Built entirely on open-weight Apache 2.0 / MIT licensed models and frameworks) [KNOWN].

22. Differentiation

23. Future Scope

1. Multi-Modal Audio/Video Maintenance Logs: Extend local multimodal capabilities to process on-device speech-to-text (Whisper local) for field technicians recording voice inspection memos [INFERENCE].
2. Fine-Tuning on Historical MRPL Schematics: Fine-tune Qwen2.5-VL adapters (LoRA) on proprietary historical ISA-5.1 P&ID symbol libraries to improve visual detection accuracy on 30-year-old scanned engineering drawings [INFERENCE].
3. Air-Gapped Multi-Node vLLM Cluster Scaling: Implement distributed model tensor parallelism across multiple GPU nodes to support 70B+ parameter open-weight reasoning models for complex multi-department refinery audits [RESEARCHED].

24. Assumptions & Open Items Log

Consolidated Assumptions

• [ASSUMPTION 1] The hackathon evaluation venue will provide or allow a student team workstation equipped with at least one NVIDIA GPU with >= 24GB VRAM (e.g., RTX 4090 / A5000) for local demo execution.
• [ASSUMPTION 2] Public open datasets for P&ID diagrams and synthetic handwritten inspection reports are sufficient to evaluate visual extraction performance during the hackathon demonstration.
• [ASSUMPTION 3] 4-bit quantized open-weight models (Q4_K_M) provide sufficient reasoning accuracy for industrial math calculations when paired with sandboxed Python code execution tools.

Consolidated Open Items & Items Needing Validation

• [NEEDS VALIDATION 1] Exact hardware procurement costs for MRPL's internal datacenter server infrastructure.
• [NEEDS VALIDATION 2] Maximum document page length for scanned PDF batch processing under single GPU VRAM constraints.
• [NEEDS VALIDATION 3] Specific MRPL corporate document template styling rules for automated .docx and .pptx generation.

Final Recommendation

The team should definitely solve this problem. It presents a clear, high-impact industrial use case with strong endorsement potential from PSU judges. The winning strategy is to build SovereignBench: a dynamic, dual-model agentic workbench combining Qwen2.5-Coder-14B and Qwen2.5-VL-7B on an air-gapped FastAPI + LangGraph backend, delivering real Word/Excel deliverables and demonstrating hardware-enforced network isolation via a live kernel-level packet audit widget.

Recommended Stack

• Frontend: React.js + Tailwind CSS + Lucide React + WebSockets
• Backend: FastAPI (Python 3.11) + LangGraph State Engine
• Model Inference: Ollama / vLLM local endpoints
• LLM Models: Qwen2.5-Coder-14B-Instruct-GGUF (Code), Qwen2.5-VL-7B-Instruct-GGUF (Vision/P&ID), DeepSeek-R1-Distill-Qwen-14B-GGUF (Reasoning)
• OCR & Ingestion: Docling (IBM) + PaddleOCR
• Vector DB & RAG: Qdrant (Local Docker) + bge-large-en-v1.5
• Sandbox & Tools: Docker Container (--net=none), python-docx, openpyxl, python-pptx
• Network Proof: Linux iptables / eBPF + tcpdump WebSocket monitor

Biggest Dependency

Availability of a single NVIDIA GPU with at least 24GB VRAM (e.g., RTX 4090 / A5000 / A6000) during the hackathon venue demonstration to host quantized model weights concurrently without VRAM out-of-memory thrashing.

Biggest Risk

VRAM Out-of-Memory (OOM) crashes during concurrent multimodal vision processing and code execution if model weight swapping is not strictly managed by the inference server.

Biggest Technical Challenge

Orchestrating a reliable multi-step ReAct agentic loop that dynamically writes Python code, executes it inside a restricted Docker sandbox, handles syntax/execution tracebacks automatically, and formats outputs into native Microsoft Office templates without human intervention.

First 5 Actions

1. Action 1: Acquire an RTX 4090 (24GB VRAM) workstation, install CUDA 12.x, Docker Engine, and Ollama/vLLM.
2. Action 2: Pre-download model weights (Qwen2.5-Coder-14B-Instruct-GGUF, Qwen2.5-VL-7B-Instruct-GGUF, bge-large-en-v1.5) into local host storage (/models/).
3. Action 3: Build and test the isolated Docker execution sandbox container (--net=none, memory-capped Python execution runtime).
4. Action 4: Implement the FastAPI + LangGraph backend with deterministic tool bindings for Docling OCR and python-docx file creation.
5. Action 5: Write the iptables script and React Network Monitor header widget to display live kernel packet statistics.

Validation Before Coding

Verify that the host GPU can maintain Qwen2.5-Coder-14B (Q4_K_M ~9GB) and Qwen2.5-VL-7B (Q4_K_M ~5GB) loaded in VRAM simultaneously with stable inference latency (<30 tokens/sec) under Ollama/vLLM.

Buildability Score /100

92 / 100 (Highly buildable for a skilled 4-6 student team using available open-weight models, open-source Python frameworks, and standard GPU hardware).

Confidence

HIGH (The solution design strictly aligns with every explicit requirement in SIH26117, respects hardware realities, avoids cloud dependencies, and provides undeniable live demo proof for PSU judging criteria).

Research Sources

1. Title: Open WebUI Documentation & Local Model Deployment
   URL: https://github.com/open-webui/open-webui
   Source Type: Official GitHub Documentation
   Supported Claim: Provides open-source UI standards for local LLM backends but lacks autonomous multi-step agentic execution loops and dynamic intent routing.
   Reliability: High (Official Repository)

2. Title: AnythingLLM Enterprise Local RAG Framework
   URL: https://github.com/Mintplex-Labs/anything-llm
   Source Type: Official GitHub Documentation
   Supported Claim: Supports air-gapped document embedding and local LLM backends, but lacks sandboxed code execution and native Office deliverable generation.
   Reliability: High (Official Repository)

3. Title: Dify LLM Application Development Platform
   URL: https://github.com/langgenius/dify
   Source Type: Official GitHub Documentation
   Supported Claim: Demonstrates drag-and-drop agent workflow orchestration capabilities.
   Reliability: High (Official Repository)

4. Title: Mangalore Refinery and Petrochemicals Limited (MRPL) Annual Report
   URL: https://admin.mrpl.co.in/img/UploadedFiles/AnnualReport/Files/161636e409d7464996e216b0ffbb5bba.pdf
   Source Type: Official PSU Enterprise Report
   Supported Claim: Verifies MRPL operational scale, IT infrastructure focus, and strict cybersecurity compliance mandates.
   Reliability: High (Official PSU Source)

5. Title: LangGraph: Building Stateful, Multi-Actor Applications with LLMs
   URL: https://github.com/langchain-ai/langgraph
   Source Type: Official Open Source Documentation
   Supported Claim: LangGraph enables stateful cyclic agent loops, tool calling, and human-in-the-loop reflection for complex workflows.
   Reliability: High (Official Framework)

6. Title: vLLM: High-throughput and Memory-efficient Inference Engine for LLMs
   URL: https://github.com/vllm-project/vllm
   Source Type: Official Technical Documentation
   Supported Claim: vLLM provides memory-efficient OpenAI-compatible local model serving on enterprise and consumer GPUs.
   Reliability: High (Official Framework)

7. Title: Qwen2.5-VL: Open-Weight Flagship Vision-Language Model
   URL: https://qwenlm.github.io/blog/qwen2.5-vl/
   Source Type: Official Model Technical Report
   Supported Claim: Qwen2.5-VL achieves state-of-the-art visual understanding on technical charts, scanned documents, and graphics layouts.
   Reliability: High (Official Model Creator)

8. Title: Qwen2.5-Coder Series Technical Overview
   URL: https://huggingface.co/collections/Qwen/qwen25-coder-6737f52f8202f5a606774a3f
   Source Type: Official Model Registry
   Supported Claim: Qwen2.5-Coder models deliver top-tier open-weight code generation and multi-file reasoning capabilities.
   Reliability: High (Official Repository)

9. Title: DeepSeek-R1 Distill Models Technical Documentation
   URL: https://github.com/deepseek-ai/DeepSeek-R1
   Source Type: Official Research Repository
   Supported Claim: DeepSeek-R1 distillations provide frontier-level chain-of-thought mathematical and logical reasoning in 14B parameter footprints.
   Reliability: High (Official Research Release)

10. Title: Docling: IBM Research Document Parsing Library
    URL: https://github.com/DS4SD/docling
    Source Type: Official Open Source Documentation
    Supported Claim: Docling parses complex PDFs, tables, and layouts locally into structured Markdown without external API dependencies.
    Reliability: High (Official Research Project)

11. Title: Qdrant Vector Search Engine Documentation
    URL: https://qdrant.tech/documentation/
    Source Type: Official Technical Documentation
    Supported Claim: Qdrant provides self-hosted, air-gapped vector storage and semantic search capabilities.
    Reliability: High (Official Vendor Documentation)

12. Title: python-docx & openpyxl Python Libraries
    URL: https://python-docx.readthedocs.io/
    Source Type: Official Open Source Documentation
    Supported Claim: Direct programmatic generation of formatted Microsoft Word and Excel files from local Python scripts.
    Reliability: High (Official Library Documentation)