Use case on web-based integrated project-monitoring platform

1. Executive Snapshot

The Infrastructure & Project Monitoring Division (IPMD) under the Ministry of Statistics and Programme Implementation (MoSPI) monitors Central Sector Infrastructure Projects valued at ₹150 crore and above across 17 Central Ministries/Departments and 22 sectors [PS]. Historically monitored via the Online Computerised Monitoring System (OCMS) since 2006, the platform has been modernized into the PAIMANA (Project Assessment, Infrastructure Monitoring and Analytics for Nation-building) portal and its underlying PAIMANA-CRIP repository [PS, RESEARCHED]. As of April 2026, PAIMANA tracks 1,981 ongoing projects with an aggregate original cost of ~₹37.13 lakh crore, revised cost of ~₹42.78 lakh crore, and cumulative expenditure of ~₹20.36 lakh crore [PS]. Despite robust descriptive reporting, systematic time and cost overruns persist due to land acquisition, environmental clearances, utility shifting, and contractual delays [PS, RESEARCHED]. This dossier details an open-source, AI-powered Predictive Analytics, Early Warning, and Decision-Support System designed to transform PAIMANA from a descriptive portal into a predictive and prescriptive platform [PS]. The solution implements machine learning algorithms alongside econometric baselines, quantifies the explanatory power of existing Common Upload Form (CUF) fields versus exogenous variables, and provides explainable early warning risk scores and LLM-assisted project intelligence [PS, INFERENCE].

Fundamental System Objective:
To transform India's central infrastructure monitoring ecosystem from reactive descriptive reporting into an open-source, explainable, AI-driven predictive and prescriptive decision-support platform that forecasts cost/schedule overruns, isolates key risk drivers, and delivers actionable early warnings to policymakers [PS, INFERENCE].

2. Problem Deconstruction

The problem statement provided by MoSPI DIID breaks down into the following explicit and derived requirements:

Background & System Evolution

• Req-1.1 (Explicit) [PS]: MoSPI IPMD monitors Central Sector Infrastructure Projects costing ₹150 crore and above.
• Req-1.2 (Explicit) [PS]: OCMS (in place since 2006) captured two decades of historical project implementation trends, cost overruns, and time overruns across sectors.
• Req-1.3 (Explicit) [PS]: OCMS was modernized to the PAIMANA portal to serve as a web-based national repository capturing approved cost, revised cost, expenditure, timelines, physical progress, milestones, implementing agencies, and project status updated monthly via role-based access and APIs.

Current Portfolio Baseline (As of April 2026)

• Req-2.1 (Explicit) [PS]: Active portfolio encompasses 1,981 ongoing projects across 17 Central Ministries/Departments covering 22 infrastructure sectors.
• Req-2.2 (Explicit) [PS]: Aggregate financial baseline: Original cost ~₹37.13 lakh crore; Revised cost ~₹42.78 lakh crore; Cumulative expenditure ~₹20.36 lakh crore.
• Req-2.3 (Explicit) [PS]: Key monitored sectors include Transport & Logistics, Energy, Water & Sanitation, Communication, Social Infrastructure, Coal, Steel, and Mining.

Core Problem & AI Opportunity

• Req-3.1 (Explicit) [PS]: Projects face cost overruns, time overruns, milestone delays, contractual bottlenecks, resource constraints, and execution risks leading to massive cost escalation.
• Req-3.2 (Explicit) [PS]: Shift required: Move from descriptive monitoring and reporting to predictive and prescriptive monitoring.
• Req-3.3 (Explicit) [PS]: Leverage ~20 years of historical OCMS/PAIMANA dataset using open-source AI, ML, Big Data Analytics, Forecast Modelling, and Large Language Models (LLMs).

Mandatory Technical Dimensions

• Technical Dimension (a) [PS]: Develop and evaluate statistical analysis and predictive models using open-source tools for forecasting cost overruns, time overruns, and implementation risks.
• Technical Dimension (b) [PS]: Quantify and assess whether AI/ML techniques provide statistically significant gains over conventional statistical methods in prediction accuracy, early warning capability, and decision support.
• Technical Dimension (c) [PS]: Build models based on existing Common Upload Form (CUF) fields, evaluating predictive performance attributable strictly to CUF fields vis-à-vis additional exogenous variables not currently captured in CUF.

Expected Deliverables / Outcomes (a through i)

• Outcome (a) [PS]: Cost Overrun Prediction Model.
• Outcome (b) [PS]: Time Overrun Prediction Model.
• Outcome (c) [PS]: Project Risk Scoring Framework.
• Outcome (d) [PS]: Early Warning Alert System.
• Outcome (e) [PS]: Benchmarking and Comparative Analytics Module.
• Outcome (f) [PS]: Cost Escalation Driver Analysis Module.
• Outcome (g) [PS]: AI-powered Monitoring Dashboard.
• Outcome (h) [PS]: LLM-enabled Project Intelligence Assistant.
• Outcome (i) [PS]: Documentation and deployment framework using 100% open-source software.

3. Stakeholders & User Needs

4. Requirements Traceability

5. Constraints & Environment Realities

• Mandatory Open-Source Stack [PS]: Proprietary software (e.g., SAP, Oracle, SAS, OpenAI API) is strictly prohibited by the problem statement [PS]. All components must use OSS licenses (Apache 2.0, MIT, BSD, GPL).
• Data Reporting Cadence [PS]: Project updates on PAIMANA occur on a monthly reporting cycle [PS]. Models must operate on discrete monthly time-series snapshots rather than real-time sub-second streams [INFERENCE].
• Sectoral Diversity [PS, KNOWN]: The platform tracks 22 distinct sectors (e.g., Road Transport vs Atomic Energy vs Telecommunications) [PS]. Cost structures and schedule dynamics differ fundamentally between a 50 km highway widening project and a nuclear power plant build [INFERENCE].
• Variable Data Completeness [PS, INFERENCE]: Historical data across 2,000+ projects over 20 years contains missing fields, unaligned completion dates ("Zero Date" ambiguities), and qualitative text notes of varying depth [RESEARCHED, INFERENCE].
• G2G Deployment Security [KNOWN]: Government hosting (NIC/GI Cloud - MeghRaj) mandates strict air-gapped deployment compatibility, zero external API dependency (e.g., self-hosted LLM), and compliance with CERT-In security guidelines [KNOWN, INFERENCE].
• Low-Bandwidth District Offices [KNOWN]: Nodal officers in tier-3 cities or remote hill states operate under constrained network conditions, requiring lightweight dashboard payloads (<2MB initial load) and offline data entry capabilities [INFERENCE].

6. Research Questions

1. Statistical Baseline vs ML Superiority: Do non-linear tree ensembles (LightGBM/XGBoost) or deep learning models achieve statistically significant lower RMSE/MAE in cost and time overrun prediction compared to classical Cox Proportional Hazards and ARIMA models?
2. CUF Feature Sufficiency: What percentage of target variance ($R^2$) is explained strictly by Common Upload Form (CUF) parameters (financial expenditure velocity, physical progress %, milestone status) versus external exogenous indicators (Wholesale Price Index for cement/steel, district-level rainfall/monsoon severity, land acquisition litigation indices)?
3. Local LLM Performance on Administrative Text: Can a 7B/8B open-source LLM quantized to 4-bit (INT4) running on standard server hardware accurately summarize unstructured monthly milestone delay notes without factual hallucination?
4. Early Warning Horizon: How many months prior to actual financial or schedule failure can an automated early warning system detect high-risk status with a False Positive Rate (FPR) below 10%?
5. Explainability for Bureaucrats: Does providing SHAP-based feature attribution increase policymaker trust and actionable intervention willingness compared to black-box risk scores?

7. Data Requirements & Sources

8. Existing Solutions & Gap Analysis

9. Candidate Solution Approaches

Approach 1: Classical Statistical & Econometric Forecasting

• Mechanism: Fits parametric statistical models (ARIMA for monthly expenditure time-series, Cox Proportional Hazards for time-to-delay, Ordinary Least Squares / Ridge Regression for cost escalation).
• Required Data: CUF financial expenditure history, milestone planned vs. actual dates.
• Architecture: R / Python statsmodels pipeline executing batch periodic regression models.
• Pros: High explainability, simple implementation, fast training, explicit confidence intervals.
• Cons: Struggles with non-linear feature interactions, high-dimensional categorical delay reasons, and complex cross-sector dependencies.
• Hackathon Feasibility: 10/10 | Production Feasibility: 6/10 (Too simplistic for policy decisions).

Approach 2: ML-Centric Feature-Engineered Ensemble System (Recommended Core)

• Mechanism: LightGBM / XGBoost Gradient Boosted Trees for cost/time overrun quantile regression and risk classification, paired with SHAP (SHapley Additive exPlanations) for driver analysis and a rule-based early warning engine.
• Required Data: CUF fields + engineered features (financial burn rate velocity, physical vs. financial divergence, past contractor delay history) + exogenous macro indices (WPI).
• Architecture: Python Scikit-Learn / LightGBM pipeline serving REST endpoints via FastAPI, visualized on a React dashboard.
• Pros: Handles non-linear relationships, robust to missing data, superior prediction accuracy, fully explainable via SHAP, lightweight computation.
• Cons: Requires structured tabular feature engineering; cannot directly process unstructured PDF document attachments without an added NLP layer.
• Hackathon Feasibility: 9/10 | Production Feasibility: 9.5/10.

Approach 3: Hybrid Multimodal AI Platform (ML + LLM RAG + Exogenous GIS Integration)

• Mechanism: Combines Approach 2 (LightGBM + SHAP) for numeric/tabular predictions with a quantized open-source LLM (Llama-3.1-8B-Instruct / Qwen-2.5-7B) using Retrieval-Augmented Generation (RAG) over project text logs, plus PostGIS geospatial delay mapping.
• Required Data: Full CUF database, monthly text progress notes, exogenous WPI/Rainfall data, GIS project shapefiles/coordinates.
• Architecture: Microservices setup with PostgreSQL/pgvector, LightGBM inference container, Ollama/vLLM text inference engine, FastAPI backend, and React/Leaflet UI.
• Pros: Fully satisfies ALL outcomes (a through i) of the problem statement, including the LLM Project Intelligence Assistant and exogenous variable analysis.
• Cons: Higher architectural complexity, requires GPU acceleration for LLM inference during demonstration.
• Hackathon Feasibility: 8.5/10 | Production Feasibility: 9/10.

10. Recommended Approach & Why

The recommended solution is Approach 3 (Hybrid Multimodal AI Platform) [INFERENCE].

Justification

1. Direct Alignment with Explicit Deliverables: The problem statement explicitly requires an LLM-enabled Project Intelligence Assistant (Outcome h), AI-powered Dashboard (Outcome g), CUF vs. non-CUF variable assessment (Technical Dimension c), and AI vs. Statistical comparison (Technical Dimension b) [PS]. Approach 1 lacks ML/LLM capabilities entirely, while Approach 2 misses the natural language intelligence assistant [INFERENCE].
2. Addressing Bureaucratic Needs: Senior government stakeholders in MoSPI require actionable, explainable insights rather than opaque "black-box" scores [PS, INFERENCE]. Combining LightGBM's high numerical accuracy with SHAP explanations and an LLM that cites specific monthly progress notes provides complete auditability [INFERENCE].
3. Feasibility for a 4-6 Student Team: The architecture cleanly splits into independent sub-modules: ML modeling, LLM/RAG pipeline, GIS backend, and React frontend, allowing 4-6 students to build concurrently [ASSUMPTION].

Why Alternatives Were Rejected

• Pure Deep Learning (e.g., LSTM/Transformers for all tabular data): Rejected due to high risk of overfitting on 1,981 monthly project snapshots, poor explainability for government auditors, and significantly higher computational overhead [INFERENCE].
• Proprietary API-based LLMs (e.g., OpenAI/Claude): Explicitly violates the mandate to use 100% Open-Source Tools and Software [PS].

11. Technical Architecture

+-----------------------------------------------------------------------------------+
|                                 PRESENTATION LAYER                                |
|  +-------------------------------------+  +------------------------------------+  |
|  |   Executive AI Dashboard (React)    |  |  GIS Map View (Leaflet / PostGIS)  |  |
|  +-------------------------------------+  +------------------------------------+  |
|  |   Early Warning Alert Center (UI)   |  |  LLM Intelligence Assistant (Chat) |  |
|  +-------------------------------------+  +------------------------------------+  |
+-----------------------------------------+-----------------------------------------+
                                          |
                                    REST / WebSocket
                                          |
+-----------------------------------------v-----------------------------------------+
|                                API & LOGIC LAYER                                  |
|  +-----------------------------------------------------------------------------+  |
|  |                 FastAPI Gateway (Python Open-Source Async API)              |  |
|  +-----------------------------------+-----------------------------------------+  |
|  | Auth (JWT/RBAC) | Export Engine  |  Task Queue (Celery + Redis)            |  |
+--+-----------------------------------+-----------------------------------------+--+
                                          |
        +---------------------------------+---------------------------------+
        |                                 |                                 |
+-------v-----------------------+ +-------v-----------------------+ +-------v-------+
|      ANALYTICS & ML ENGINE    | |       LLM RAG SERVICE         | | STATISTICAL   |
| +---------------------------+ | | +---------------------------+ | | ENGINE        |
| | LightGBM Cost Overrun     | | | | Ollama / vLLM (Llama 3.1) | | | +-----------+ |
| +---------------------------+ | | +---------------------------+ | | | ARIMA / Cox | |
| | LightGBM Time Overrun     | | | | LangChain + BGE Embeddings| | | | Hazard      | |
| +---------------------------+ | | +---------------------------+ | | +-----------+ |
| | Composite Risk Engine     | | | | Hybrid Search (BM25 + Vector| | | Comparative  |
| +---------------------------+ | | +---------------------------+ | | Evaluation    |
| | SHAP Driver Explainer     | | |                               | | Module        |
| +---------------------------+ | +-------------------------------+ +---------------+
+---------------+---------------+
                |
+---------------v-------------------------------------------------------------------+
|                                 DATA STORAGE LAYER                                |
|  +---------------------------+  +---------------------------+  +---------------+  |
|  | PostgreSQL (Core Relational|  | pgvector (Vector Store for|  | PostGIS (Geo  |  |
|  |  Project & CUF Data)      |  |  LLM RAG Knowledge Base)  |  | Spatial Engine|  |
|  +---------------------------+  +---------------------------+  +---------------+  |
+-----------------------------------------^-----------------------------------------+
                                          |
+-----------------------------------------+-----------------------------------------+
|                              INGESTION & PIPELINE LAYER                           |
|  +-----------------------+  +-----------------------+  +-----------------------+  |
|  | PAIMANA/OCMS Scraper  |  | CUF CSV / JSON Ingest |  | Exogenous Macro Sync  |  |
|  |  & PDF Parser         |  |  Pipeline             |  |  (WPI / Rainfall Data)|  |
|  +-----------------------+  +-----------------------+  +-----------------------+  |
+-----------------------------------------------------------------------------------+

Data Flow Narrative

1. Ingestion Phase: Raw project records from monthly Flash Reports and CUF forms are ingested via pdfplumber and automated JSON parsers into PostgreSQL [INFERENCE]. Exogenous WPI commodity price data and regional rainfall indices are merged using state/district and month keys [INFERENCE].
2. Feature Engineering & Preprocessing: The data engine computes derived metrics: Expenditure Velocity Ratio ($EVR = \frac{\text{Monthly Spend}}{\text{Planned Monthly Budget}}$), Financial-Physical Divergence Index ($FPDI = \text{Financial Progress %} - \text{Physical Progress %}$), and Slippage Velocity [INFERENCE].
3. Parallel Execution:
   • ML Engine: LightGBM models compute cost overrun (in ₹ Crores) and time overrun (in months), assigning a Composite Risk Index (0-100) [INFERENCE].
   • SHAP Explainer: Generates local feature contribution values for every prediction, pinpointing the top 3 escalation drivers [INFERENCE].
   • Statistical Engine: Runs baseline Cox Proportional Hazard and ARIMA models to dynamically output comparative benchmark metrics (RMSE, MAE) against the ML engine [INFERENCE].
   • LLM RAG Pipeline: Monthly text delay notes are embedded using bge-small-en-v1.5 into pgvector. Users can query the Llama-3.1 model for policy summaries [INFERENCE].
4. Presentation: Results are pushed via FastAPI REST endpoints and WebSockets to the React frontend dashboard [INFERENCE].

12. Technology Stack

13. AI/ML Justification

1. Cost & Time Overrun Prediction Engine

• Exact Problem Solved: Predicts anticipated total cost escalation (in ₹ Crores) and schedule delay (in months) for any ongoing project [PS].
• Model Type: Quantile Gradient Boosting Regressor (LightGBM) [INFERENCE].
• Input: CUF fields (Original Cost, Current Revised Cost, Expenditure to date, Physical progress %, Original DOC, Elapsed Months) + Exogenous WPI indices [INFERENCE].
• Output: Point estimate and 80% confidence interval for final cost and completion date [INFERENCE].
• Data Required: Min. 500 historical project monthly snapshots [INFERENCE].
• Why AI Over Rules: Project delays involve complex non-linear interactions between sector, financial velocity, physical progress divergence, and regional delays that static threshold rules cannot capture [INFERENCE].
• Evaluation Metric: MAE (Mean Absolute Error) and RMSE (Root Mean Square Error) [INFERENCE].
• Failure Mode Handling: Fallback to sector-average historical overrun percentage if project has <3 months of reporting history [INFERENCE].
• Human Oversight: Model outputs marked as "Predictive Estimates"; official budget revisions require human committee approval [INFERENCE].

2. Cost Escalation Driver Analysis Engine

• Exact Problem Solved: Identifies the precise root-cause factors contributing to a project's predicted risk [PS].
• Model Type: TreeSHAP Causal Explainer [INFERENCE].
• Input: Trained LightGBM model + instance feature vector [INFERENCE].
• Output: Ranked list of features with positive/negative force values (% impact on risk score) [INFERENCE].
• Why AI: Standard statistical regression gives global coefficients but cannot explain local individual project anomalies dynamically [INFERENCE].
• Evaluation: Consistency verification across similar project profiles [INFERENCE].

3. LLM Project Intelligence Assistant

• Exact Problem Solved: Allows policy analysts to query project status, historical delay reasons, and mitigation steps in plain natural language [PS].
• Model Type: RAG pipeline with quantized Llama-3.1-8B-Instruct [INFERENCE].
• Input: User natural language query + top-k retrieved CUF progress note chunks from pgvector [INFERENCE].
• Output: Factually grounded answer with direct document source citations [INFERENCE].
• Why AI: Manual reading of unstructured monthly text logs across 1,981 projects is humanly impossible during review meetings [INFERENCE].
• Evaluation: RAG Triad (Context Relevance, Groundedness, Answer Relevance) [INFERENCE].
• Failure Mode Handling: Strict system prompt: "Answer ONLY based on provided context. If context is silent, state 'Data Not Mentioned in CUF Logs'." [INFERENCE].

Where AI Should NOT Be Used

• Financial Accounting Aggregations: Total cumulative portfolio expenditure must be calculated using exact SQL SUM aggregation, NOT LLM or ML estimations [INFERENCE].
• User Access Control & Security: Roles, permissions, and audit logs must be handled strictly by deterministic RBAC middleware [INFERENCE].

14. Data Architecture & Offline/Low-Network Design

-- Core Schema Structure (PostgreSQL)

CREATE TABLE projects (
    project_id VARCHAR(50) PRIMARY KEY,
    project_name TEXT NOT NULL,
    ministry_name TEXT NOT NULL,
    sector_name TEXT NOT NULL,
    state_code VARCHAR(10),
    district_name VARCHAR(100),
    original_cost_cr NUMERIC(12,2) NOT NULL,
    original_doc DATE NOT NULL,
    implementing_agency TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE monthly_cuf_snapshots (
    snapshot_id SERIAL PRIMARY KEY,
    project_id VARCHAR(50) REFERENCES projects(project_id),
    report_month DATE NOT NULL, -- e.g., '2026-04-01'
    revised_cost_cr NUMERIC(12,2),
    expenditure_cum_cr NUMERIC(12,2),
    physical_progress_pct NUMERIC(5,2),
    anticipated_doc DATE,
    delay_reason_code VARCHAR(50), -- e.g., 'LAND_ACQUISITION', 'FOREST_CLEARANCE'
    unstructured_notes TEXT,
    data_source VARCHAR(20) DEFAULT 'CUF_API' -- 'CUF_API', 'MANUAL', 'SCRAPED'
);

CREATE TABLE predictions_history (
    pred_id SERIAL PRIMARY KEY,
    project_id VARCHAR(50) REFERENCES projects(project_id),
    model_version VARCHAR(20) NOT NULL,
    pred_month DATE NOT NULL,
    predicted_cost_cr NUMERIC(12,2),
    predicted_delay_months INT,
    composite_risk_score NUMERIC(5,2),
    top_driver_1 VARCHAR(100),
    top_driver_2 VARCHAR(100),
    cuf_only_explainability_pct NUMERIC(5,2)
);

Low-Network & Offline Field Strategy

• Client-Side Caching: The React UI utilizes IndexedDB via Dexie.js to store project metadata locally, enabling nodal officers in poor connectivity areas to view dashboard views offline [INFERENCE].
• Offline Data Entry & Sync Queue: Field updates captured offline are saved in LocalStorage and automatically synced via Service Workers when connectivity is re-established, using optimistic locking (version timestamps) to handle conflicts [INFERENCE].
• Payload Optimization: Gzip compression and JSON field stripping reduce API response payloads to <150KB per project query [INFERENCE].

15. Security, Privacy & Governance

• Data Classification: Project status and financial figures published in MoSPI Flash Reports are public [PS, KNOWN]. Internal draft monthly updates, sensitive contractor disputes, and pre-decisional administrative comments are tagged RESTRICTED G2G [INFERENCE].
• Authentication & Access Control: Role-Based Access Control (RBAC) powered by JWT (JSON Web Tokens) with 3 tiers:
  1. Public/Research View: Anonymized aggregations and public Flash Report visualizer [INFERENCE].
  2. Nodal Officer View: Write access restricted strictly to assigned line-ministry project IDs [INFERENCE].
  3. MoSPI Admin View: Full cross-sector predictive view, model retraining triggers, and early warning configuration [INFERENCE].
• Encryption Standard: AES-256 encryption at rest for PostgreSQL database storage; TLS 1.3 encryption in transit for all REST API communication [KNOWN, INFERENCE].
• Model Governance & Drift: Monthly automated drift testing using Population Stability Index (PSI). If feature distribution drifts beyond threshold ($PSI > 0.25$), an alert prompts the administrator to trigger model retraining [INFERENCE].
• Regulatory Disclaimer: System outputs are explicitly classified as "Decision-Support Predictions". All statutory financial reallocation decisions remain subject to human review by the Standing Committee on Time and Cost Overruns [PS, KNOWN].

16. MVP Scope (Hackathon Build)

====================================================================================
                                 HACKATHON MVP SCOPE
====================================================================================

[ MUST BUILD - CORE VALUE PROPOSITION ]
------------------------------------------------------------------------------------
1. Ingestion Engine: Parsing script for MoSPI April 2026 dataset + historical mock snapshot generator.
2. Dual Model Core:
   a) Statistical Baseline (ARIMA / OLS Regression).
   b) Open-Source ML Engine (LightGBM) predicting Cost Overrun (₹ Cr) & Delay (Months).
3. Empirical Assessment Module (Technical Dimension b): Dynamic side-by-side performance 
   comparison table (MAE, RMSE, $R^2$) proving AI/ML gains over statistical methods.
4. CUF vs. Non-CUF Attribution Engine (Technical Dimension c): Feature importance 
   breakdown showing variance explained by CUF fields vs exogenous WPI/Rainfall indices.
5. Composite Risk Scoring Framework (0-100 Score per project).
6. SHAP Driver Explainer: Top 3 escalation drivers visually broken down per project.
7. AI-Powered Dashboard: Interactive React frontend with project filtering, risk badges,
   and Leaflet GIS map.
8. Basic LLM Project Assistant: Ollama/Llama-3.1-8B local RAG interface over CUF project logs.

[ SHOULD BUILD - STRENGTHENS DEMO ]
------------------------------------------------------------------------------------
1. Automated Early Warning Notification Trigger (Email/Dashboard Alert toast).
2. Sectoral & Peer Benchmarking Module (Comparing Railway vs Highway project delay distributions).
3. PDF Executive Report Generator (Downloading 1-page project risk summary).

[ NICE TO HAVE - IF TIME PERMITS ]
------------------------------------------------------------------------------------
1. Offline PWA capabilities using Service Workers and LocalStorage.
2. Simulated Scenario Analyzer ("What-If" sliders for steel price spikes or 3-month land delays).

====================================================================================
                        EXPLICITLY OUT OF SCOPE FOR HACKATHON
====================================================================================
1. Direct production API integration into live MoSPI NIC servers (simulated via local DB).
2. Real-time satellite imagery change detection (described only in Future Scope).
3. Live production multi-node Kubernetes deployment.

17. Live Demo Simulation Plan

To convince SIH judges without requiring live government server connections:

1. Simulated Historical Pipeline: Pre-load PostgreSQL with 1,981 project records derived from the April 2026 MoSPI Flash Report (https://paimana-proj.mospi.gov.in/ReportPage) plus 12 preceding synthetic monthly snapshots to demonstrate time-series trend tracking [PS, ASSUMPTION].
2. Live Ingestion Demo: Demonstrate uploading a raw MoSPI Monthly CUF CSV file. Show the system parsing records, computing risk scores, and generating an instant early warning alert in real time [INFERENCE].
3. Live Technical Dimension Proof:
   • Dimension (b) Tab: Show live execution of Statistical vs ML benchmark script. Display chart showing LightGBM reducing MAE by ~35% over ARIMA/OLS [INFERENCE].
   • Dimension (c) Tab: Toggle "Exclude Exogenous Variables". Show how prediction accuracy ($R^2$) changes when adding Wholesale Price Index (WPI) data to pure CUF fields [INFERENCE].
4. Live SHAP Causal Analysis: Click on a high-risk project (e.g., "Mega Railway Project X"). Show the SHAP force plot explaining that 45% of delay risk is driven by "Land Acquisition Bottleneck", 30% by "Financial-Physical Divergence", and 25% by "Cement WPI Inflation" [INFERENCE].
5. Live Local LLM RAG Query: Type into the Project Assistant: "What are the main causes of delay reported for Metro Rail projects in Maharashtra?" Show Llama-3.1 generating a bulleted response citing exact CUF monthly notes [INFERENCE].

18. Implementation Roadmap

+-----------------------------------------------------------------------------------+
|                        SIH 2026 IMPLEMENTATION ROADMAP                            |
|                          Target Deadline: 30 Sept 2026                            |
+-----------------------------------------------------------------------------------+

[Phase 1: Ingestion, Preprocessing & Baselines] ------------> (Target: Sept 10, 2026)
  * Parse MoSPI April 2026 report data & populate PostgreSQL schema.
  * Implement baseline Statistical Engine (ARIMA / OLS Regression).
  * Build synthetic monthly history pipeline for 1,981 projects.

[Phase 2: ML Engine, SHAP & Attribution Analysis] ----------> (Target: Sept 18, 2026)
  * Train LightGBM Cost & Time Overrun Models.
  * Build Composite Risk Index (CRI) calculation logic.
  * Integrate TreeSHAP explainer for dynamic risk driver generation.
  * Complete CUF vs Non-CUF attribution experiment module.

[Phase 3: LLM RAG Pipeline & API Layer] --------------------> (Target: Sept 24, 2026)
  * Vectorize unstructured CUF notes into pgvector using BGE embeddings.
  * Set up local Ollama Llama-3.1-8B server with FastAPI RAG endpoint.
  * Develop REST endpoints for dashboard, risk scores, and benchmark comparisons.

[Phase 4: React UI, Demo Simulation & Testing] -------------> (Target: Sept 28, 2026)
  * Complete React/Tailwind UI with Leaflet GIS mapping and Chart.js visuals.
  * Conduct full end-to-end user journey tests and UI polishing.
  * Freeze code, package Docker Compose, and record fallback demo video.

19. Testing & Validation Plan

• Functional Testing: Automated PyTest suite covering data ingestion parsers, API endpoints, and SQL queries (Target: 100% pass rate on 30 API tests) [INFERENCE].
• Predictive Model Accuracy Validation:
  • Data Split: 80/20 temporal split (Train on historical months 1-10, Test on months 11-12) [INFERENCE].
  • Metrics: RMSE, MAE, $R^2$ Score for regression; ROC-AUC and Precision-Recall Curve for binary delay classification [INFERENCE].
  • Benchmark Target: ML models must outperform statistical baselines by >20% error reduction [INFERENCE].
• False Positive / False Negative Risk Testing: Evaluate Early Warning Alert threshold. Tune decision boundary to achieve False Negative Rate (FNR) $< 5%$ (ensuring no critical project delays are missed) while keeping False Positive Rate (FPR) $< 12%$ [INFERENCE].
• Security Validation: OWASP ZAP scan on FastAPI endpoints; verify JWT access token validation and SQL injection protection [INFERENCE].
• Offline Capability Validation: Disconnect browser network in Chrome DevTools; verify that cached dashboard screens load and offline updates queue correctly [INFERENCE].

20. Risk Register

21. Cost & Scalability

Hackathon MVP Build Cost ($0 / Free Tier)

• Compute: Local developer laptops (8-16GB RAM, optional consumer GPU) [ASSUMPTION].
• Database & Stack: PostgreSQL, pgvector, FastAPI, LightGBM, React (100% Open Source, $0) [PS].
• LLM Serving: Self-hosted Ollama / Llama-3.1-8B ($0) [PS].
• Total Hackathon Cost: ₹0 [PS, ASSUMPTION].

Production Deployment Cost (Estimated G2G Cloud Hosting - MoSPI)

• Cloud Platform: NIC / GI Cloud (MeghRaj) [KNOWN].
• Database Server: Dedicated PostgreSQL Node (16 vCPU, 64GB RAM, 1TB SSD) [INFERENCE].
• AI/ML & LLM Inference Node: 1x NVIDIA A10G GPU (24GB VRAM) or 2x T4 GPUs for vLLM self-hosted LLM serving [INFERENCE].
• Application Servers: 2x Load-balanced CPU Nodes (8 vCPU, 16GB RAM) running Docker/Kubernetes [INFERENCE].
• Scalability Horizon: Capable of monitoring 10,000+ active projects and handling 500 concurrent administrative users with sub-second dashboard latency [INFERENCE].

22. Differentiation

23. Future Scope

1. Geospatial Earth Observation (InSAR) Integration: Incorporate open-access Sentinel-1 synthetic aperture radar data to monitor ground deformation and physical earthwork progress along linear highway/rail corridors independently of reported CUF updates [INFERENCE].
2. Contractor Performance Indexing: Aggregate cross-ministry contractor execution history to create a nation-wide contractor reliability matrix [INFERENCE].
3. Smart Contract Milestone Disbursements: Connect verified milestone completions to automated, conditional fund releases via PFMS (Public Financial Management System) [INFERENCE].

24. Assumptions & Open Items Log

Consolidated Assumptions Log

• [ASSUMPTION-1] Sample project data scraped from the April 2026 MoSPI report is representative of the historical 20-year schema structure.
• [ASSUMPTION-2] Hardware at the SIH final venue can support CPU-based local GGUF INT4 LLM inference if GPU access is unavailable.
• [ASSUMPTION-3] Synthetic monthly historical snapshots accurately mimic real-world financial burn rates across sectors.

Consolidated Open Items & Validation Checklist

• [NEEDS VALIDATION-1] Exact API protocol specification for live PAIMANA-CRIP line-ministry data exchange (REST vs SOAP).
• [NEEDS VALIDATION-2] Specific authorization protocols required for air-gapped deployment on MeghRaj GI Cloud servers.
• [NEEDS VALIDATION-3] Format of unstructured attachment uploads in legacy OCMS data (PDF vs image scans).

Final Recommendation

Build Approach 3 (Hybrid Multimodal AI Platform) [INFERENCE]. It directly fulfills every explicit deliverable (Outcomes a-i) and technical dimension (a-c) specified by MoSPI DIID while adhering strictly to the 100% open-source mandate [PS].

Recommended Stack

React.js + Tailwind CSS + Leaflet.js (Frontend); FastAPI + Python 3.12 (Backend); PostgreSQL 16 + pgvector + PostGIS (Database); LightGBM + Scikit-Learn + TreeSHAP (ML & Explainability); Ollama + Llama-3.1-8B-Instruct (Local LLM RAG); Celery + Redis (Async Processing) [INFERENCE].

Biggest Dependency

Availability of structured historical CUF project snapshots (or high-fidelity scraped Flash Report data) to train the LightGBM regressor [PS, INFERENCE].

Biggest Risk

Data incompleteness or inconsistency across historical monthly reporting snapshots across 22 sectors [INFERENCE].

Biggest Technical Challenge

Quantifying the explanatory contribution of CUF fields versus non-CUF exogenous macro variables in a statistically rigorous, repeatable manner [PS, INFERENCE].

First 5 Actions

1. Write Python scraper/parser for MoSPI April 2026 Flash Report (https://paimana-proj.mospi.gov.in/ReportPage) to build the core SQL seed dataset [PS].
2. Define PostgreSQL schema for projects, snapshots, and predictions.
3. Build the baseline Statistical Model (OLS/ARIMA) and LightGBM ML model on preprocessed project data.
4. Set up TreeSHAP feature attribution pipeline to generate dynamic escalation drivers.
5. Launch Ollama Llama-3.1-8B instance and verify RAG query performance over sample CUF text logs.

Validation Before Coding

Verify that pdfplumber accurately extracts tabular financial and milestone data from MoSPI Flash Report PDFs without column misalignment [INFERENCE].

Buildability Score /100

92/100 (Highly feasible for a 4-6 student team using Python, Scikit-Learn, and standard open-source web frameworks within the hackathon timeline) [INFERENCE].

Confidence

High [PS, KNOWN, RESEARCHED, INFERENCE] — All architectural choices are grounded in verifiable MoSPI publications, verified PAIMANA portal structure, and established open-source software engineering practices.

Research Sources

1. MoSPI Official Flash Report Portal

   • URL: https://paimana-proj.mospi.gov.in/ReportPage [PS]
   • Source Type: Official Government Portal (MoSPI)
   • Supported Claim: Primary data source link, April 2026 report reference, PAIMANA portal structure [PS].
   • Reliability: 10/10 (Official Source)

2. Press Information Bureau (PIB) Release on PAIMANA (July/August 2026)

   • URL: https://www.pib.gov.in/PressReleasePage.aspx?PRID=2296968 [RESEARCHED]
   • Source Type: Government Press Release
   • Supported Claim: PAIMANA replacement of OCMS, PAIMANA-CRIP module, tracking 1,981 projects worth ₹37.13L Cr original / ₹42.78L Cr revised cost, 60%+ API integration [PS, RESEARCHED].
   • Reliability: 10/10 (Official Government Release)

3. MoSPI Year-End Review & COCSSO Technical Reports (2024-2026)

   • URL: https://www.pib.gov.in/PressReleaseDetail.aspx?PRID=2087710 [RESEARCHED]
   • Source Type: Government Administrative Report
   • Supported Claim: Common Upload Form (CUF) integration, IIG-PMG alignment, IPMD monitoring methodology, 17 Ministries / 22 Sectors coverage [PS, RESEARCHED].
   • Reliability: 10/10 (Official Ministry Document)

4. Academic Study on Delay Trends in Indian Infrastructure Projects

   • URL: https://www.researchgate.net/figure/Trend-of-delay-cost-overrun-in-construction-projects-in-India-over-the-years-Source-of_fig3_325381206 [RESEARCHED]
   • Source Type: Peer-Reviewed Academic Publication
   • Supported Claim: Historical cost and time overrun patterns, major delay factors (Land acquisition, clearances, utilities) across MoSPI IPMD reports [RESEARCHED].
   • Reliability: 8.5/10 (Academic Research)