From what to build to how to build it — our platform helps you understand, plan, and solve SIH 2026 problem statements with complete insights into requirements, solution approaches, technology stacks, architecture, AI/ML, and implementation roadmaps.
Search for your problem statement, and start building your solution.
| Description: | Background: 1.1 The Clinical History-Taking Bottleneck in Indian Hospitals History taking - the structured elicitation of a patient's presenting complaints, history of present illness, past medical and surgical history, drug and allergy history, family and personal history, and a review of systems - is the single most important diagnostic activity in clinical medicine. Classical teaching holds that a well-conducted history yields the correct diagnosis in 70–80% of cases, even before examination or investigation. Yet in India's overburdened public hospital outpatient departments (OPDs), the time available for this critical interaction has collapsed to unsustainable levels. India operates one of the most patient-dense healthcare systems in the world. Tertiary government hospitals and apex institutions routinely register 4,000–10,000 OPD patients per day, with a doctor-to-patient consultation time frequently reported between 2 and 5 minutes - among the shortest globally (study published in BMJ Open, 2017, across 67 countries placed India's average primary-care consultation at just over 2 minutes). Within this window, the physician must simultaneously elicit history, examine the patient, review prior records, formulate a diagnosis, counsel, and prescribe. The result is systematic under-elicitation of history, missed comorbidities, repeated questioning across visits, and diagnostic error. AYUSH institutions face an additional layer of complexity. Ayurvedic history taking (Trividha, Ashtavidha, and Dashavidha Pariksha) requires detailed assessment of Prakriti (constitution), Vikriti (current imbalance), Agni (digestive capacity), Koshtha (bowel nature), Ahara-Vihara (diet and lifestyle), Nidana (causative factors), and Samprapti (pathogenesis) - a far more extensive history framework than allopathic intake. Capturing this depth manually within OPD time constraints is effectively impossible, forcing practitioners to abbreviate the very assessment that defines personalized Ayurvedic care. 1.2 The Documentation and Records Fragmentation Problem Compounding the time problem is the fragmentation of patient records. Patients in India typically carry physical paper prescriptions, laboratory reports, discharge summaries, and imaging films from multiple prior providers. During consultation, the physician must manually scan through these unstructured documents - often handwritten, in varying languages, and chronologically disordered - consuming a significant fraction of the already-scarce consultation time. There is no point-of-entry mechanism to digitize, structure, and chronologically organize a patient's prior medical documents before they reach the consultation room. The Ayushman Bharat Digital Mission (ABDM) has established the national digital health infrastructure - ABHA (Ayushman Bharat Health Account) IDs, the Health Information Exchange, and FHIR-based interoperability standards. However, the 'first-mile' problem remains unsolved: there is no efficient, patient-facing software platform that captures structured history and digitizes documents into the ABDM ecosystem before the clinical encounter begins. 1.3 The Opportunity: AI-Powered Digital Clinical Intake Platform Self-service kiosks have transformed high-throughput service industries - ATMs in banking, self-check-in terminals in aviation, and ordering kiosks in quick-service restaurants - by offloading structured data-entry tasks from human staff to the user, dramatically improving throughput and accuracy. In healthcare, patient check-in kiosks are now widespread in developed-country hospitals, but these are limited to administrative check-in. None perform deep, AI-driven, multimodal clinical history acquisition with medical document digitization. The convergence of mature enabling technologies - robust automatic speech recognition (ASR) for Indian languages and accents (Bhashini / AI4Bharat models), large language models for conversational clinical history elicitation, high-accuracy OCR for handwritten and printed medical documents, and ABDM's FHIR interoperability - now makes it feasible to build an AI-powered clinical history software platform. Description: 2.1 The Problem in Precise Terms There is no purpose-built, patient-facing software platform that enables patients to independently and comprehensively record their medical history - through both natural spoken conversation and guided touchscreen interaction - and simultaneously digitize their existing physical medical documents, generating a structured, physician-ready clinical history summary that integrates with the hospital information system and the ABDM ecosystem before the patient enters the consultation room. 2.2 Why Existing Solutions Fall Short • Existing hospital registration systems (currently deployed in some Indian hospitals) capture only demographic and appointment data - name, age, department, token number. They do not elicit any clinical history or process medical documents. • Mobile health apps and tele-triage chatbots require smartphone literacy, stable connectivity, and patient enrolment ahead of the visit - excluding the large elderly, rural, low-literacy, and first-visit patient populations who form the bulk of government hospital OPD load. • Manual nurse-led triage / history desks are themselves human-resource-limited, do not scale to 5,000+ daily patients, and reintroduce the same time and transcription bottleneck the system is trying to eliminate. • Generic document scanners digitize images but do not extract, structure, or chronologically organize clinical content, nor link it to a structured history or ABHA record. 2.3 Specific Challenges a Solution Must Overcome • Multilingual, multi-accent voice capture in noisy hospital environments across Hindi, English, and major regional languages, for patients of varying literacy and digital comfort. • Accessibility for low-literacy and elderly users through intuitive icon-driven UI, audio prompts, and conversational guidance - the software platform must be usable by a first-time, non-tech-savvy patient with zero training. • Accurate clinical history structuring converting free-form patient narration into a standardized, physician-readable history (chief complaint, HPI, past history, drug/allergy, family, personal, review of systems) - and, for AYUSH settings, Dashavidha Pariksha parameters. • Reliable medical document digitization OCR of handwritten and printed prescriptions, lab reports, and discharge summaries in multiple languages, with intelligent extraction of diagnoses, medications, and investigation values. • Privacy, consent, and data security compliance with the Digital Personal Data Protection Act 2023 and ABDM consent framework - handling sensitive health data within a secure software environment. Expected solution: 3.1 Solution Overview - 'MediKiosk' AI Clinical History Software Platform The proposed solution - tentatively designated MediKiosk - a software platform for an AI-powered clinical history software platform that allows any patient to record a comprehensive medical history through natural voice conversation and guided touchscreen interaction, scan and digitize their existing physical medical documents, and generate a structured, physician-ready clinical history summary that is pushed to the hospital information system (HIS) and linked to the patient's ABHA record - all completed before the consultation, with minimal staff assistance required. • Insert Table*3.2 3.3 Software & AI Stack (Integrated) Module A - Conversational Multimodal History Engine A conversational AI engine that conducts a structured clinical history interview through both voice and touch. The patient speaks naturally in their preferred language; the engine asks intelligent follow-up questions (e.g., on stating 'chest pain', it probes onset, character, radiation, aggravating/relieving factors - the SOCRATES framework) and simultaneously offers touch-based multiple-choice options for patients who prefer tapping. Built on Indian-language ASR, a dialogue manager constrained by a clinical history ontology, and text-to-speech for audio prompts. • Adaptive questioning: dynamically branches based on chief complaint and prior answers, mirroring a physician's clinical reasoning to elicit a complete HPI and review of systems • Dual-mode input: every question answerable by speaking OR tapping, ensuring usability across literacy and comfort levels • AYUSH history mode: for Ayurvedic OPDs, an extended interview capturing Dashavidha Pariksha (Prakriti, Vikriti, Sara, Samhanana, Pramana, Satmya, Sattva, Ahara Shakti, Vyayama Shakti, Vaya) and Ahara-Vihara assessment • Red-flag detection: AI flags emergency symptoms (e.g., acute chest pain with dyspnoea, stroke symptoms) and triggers immediate priority alert to triage staff rather than routine queueing Module B - Medical Document Digitization & Intelligence An integrated scanning and document-AI pipeline that allows the patient to upload prior prescriptions, lab reports, and discharge summaries. The system performs high-accuracy OCR (printed and handwritten, multilingual), then extract and structure clinical entities. • Intelligent extraction: diagnoses, prescribed medications with dosages, investigation results with values and reference ranges, and procedure/surgery history • Chronological organization: automatically dates and orders documents into a coherent medical timeline for the physician • Abnormal-value highlighting: flags out-of-range lab values and potential drug interactions for physician attention Module C - Structured History Summary Generator An AI summarization engine that synthesizes the conversational history and the digitized documents into a single, concise, physician-ready clinical summary in standard format - presented on the consultation screen the moment the patient enters the room. The physician reads a complete, structured history in seconds rather than spending minutes eliciting it, and can edit/confirm before saving. • Standard clinical format: Chief complaint ? HPI ? Past medical/surgical ? Drug & allergy ? Family ? Personal ? ROS ? Prior investigations summary • Editable & verifiable: physician retains full control - the summary is a draft to accept, amend, or reject, never an autonomous diagnosis • Bilingual output: patient-facing audio confirmation in local language; physician-facing summary in English/Hindi Module D - Consent, Privacy & ABDM Integration A robust consent and security layer compliant with the Digital Personal Data Protection Act 2023 and the ABDM consent framework. The patient authenticates via ABHA ID, grants explicit consent for data capture and sharing, and the structured history is pushed to the hospital HIS/EMR and linked to the ABHA Personal Health Record via FHIR APIs. • Secure processing: voice and document AI are processed securely within the software platform • Session termination: temporary session data is cleared immediately after submission • Consent-first design: granular, revocable consent with audio explanation for low-literacy patients 3.4 End-to-End Patient Journey • Step 1 - Identify: Patient logs into the software platform, enters/scans ABHA ID or Aadhaar details or registers as new; selects language; grants consent (audio-guided) • Step 2 - Converse: AI conducts adaptive voice + touch history interview, capturing chief complaint, HPI, and full history; red flags trigger priority triage • Step 3 - Scan: Patient uploads prior prescriptions, lab reports, and discharge summaries; AI digitizes, structures, and timelines them • Step 4 - Summarize & Route: AI generates structured history summary, links to ABHA, pushes to HIS, updates the patient's digital record; summary appears on physician's screen at consultation • Step 5 - Consult: Physician reviews complete history in seconds, edits/confirms, and devotes the full consultation to examination, reasoning, and counselling |
| Organization: | Ministry of Ayush |
| Department: | All India Institute of Ayurveda |
| Category: | Software |
| Theme: | MedTech / BioTech / HealthTechtd> |
| Youtube Link: | Not provided |
| Dataset Link: | Additional Information regarding PS https://drive.google.com/file/d/1mQ6Qp2MKL8JXdL2kJYqV-SFqcfbxSvrd/view?usp=drive_link |
| Contact info: | Not provided |
The Indian healthcare system suffers from extreme outpatient department (OPD) congestion in government tertiary centers, registering 4,000 to 10,000 patients daily [PS]. This high volume restricts doctor-patient consultation times to an average of 2 to 5 minutes [PS] (with primary care consultations averaging ~2 minutes as documented in BMJ Open 2017) [RESEARCHED]. Within this brief window, clinicians must take a comprehensive history, conduct physical examinations, review scattered physical records, and formulate diagnostic and treatment plans [PS]. The problem is compounded in Ayurvedic institutions (such as the All India Institute of Ayurveda), where clinical history requires extensive assessment across Trividha, Ashtavidha, and Dashavidha Pariksha frameworks [PS]. Furthermore, historical patient data remains trapped in unstructured, handwritten paper records [PS].
MediKiosk is a multimodal, multilingual, patient-facing digital intake and document digitization software platform designed to run on self-service hospital kiosks and tablets [PS]. Using Indian-language Automatic Speech Recognition (ASR via Bhashini/AI4Bharat), a constrained clinical dialogue engine, high-accuracy handwritten medical OCR, and ABDM FHIR R4 interoperability, MediKiosk captures structured medical histories and digitizes prior paper records before the patient enters the consultation room [PS]. This transforms unstructured intake into a physician-ready clinical summary, restoring the consultation room for examination and clinical reasoning [PS].
| Field | Value |
|---|---|
| Problem Statement ID | SIH26047 [PS] |
| Title | Patient Case-Taking Software [PS] |
| Organization | Ministry of Ayush [PS] |
| Department | All India Institute of Ayurveda (AIIA) [PS] |
| Category | Software [PS] |
| Theme | MedTech / BioTech / HealthTech [PS] |
| Submission Deadline | 30 September 2026 [PS] |
| Ideas Submitted Counter | 4 / 500 [PS] |
| Dataset Link | https://drive.google.com/file/d/1mQ6Qp2MKL8JXdL2kJYqV-SFqcfbxSvrd/view?usp=drive_link [PS] |
To provide an accessible, multimodal, voice-and-touch AI platform that empowers low-literacy and non-tech-savvy patients to independently record structured clinical histories and digitize legacy health records prior to consultation, generating FHIR R4-compliant, physician-ready summaries linked to ABHA [PS].
The problem description is deconstructed into atomic requirements preserving all explicit statements [PS]:
| Requirement ID | Source Text / Explicit Requirement | Interpretation | Implementation Implication | Confidence |
|---|---|---|---|---|
| REQ-01 | "Time available for history-taking in OPDs collapsed to unsustainable levels (2–5 mins)... BMJ Open 2017 study placed India average at just over 2 mins." | System must reduce in-consultation data entry to under 30 seconds by offloading intake prior to entry. | Pre-consultation kiosk processing; summary must be scannable in <15 seconds. | [PS] |
| REQ-02 | "AYUSH history taking requires detailed assessment of Prakriti, Vikriti, Agni, Koshtha, Ahara-Vihara, Nidana, and Samprapti... Dashavidha Pariksha parameters." | Platform must natively support Ayurvedic diagnostic taxonomies alongside allopathic review of systems. | Dual-mode clinical schema featuring Dashavidha Pariksha (Prakriti, Vikriti, Sara, Samhanana, etc.) & NAMASTE/ICD-11 TM2 coding. | [PS] |
| REQ-03 | "Patients carry physical paper prescriptions, lab reports, discharge summaries... unstructured, handwritten, varying languages, chronologically disordered." | System must ingest physical paper documents via scanner/camera and structure them automatically. | Vision OCR pipeline trained on handwritten Indian clinical script, extracting entity dates, drugs, and values into a timeline. | [PS] |
| REQ-04 | "No point-of-entry mechanism to digitize, structure, and chronologically organize prior medical documents before reaching consultation." | Scanning and OCR execution must happen at point-of-entry (kiosk terminal) prior to queuing. | Hardware UI integration with document feed scanner/camera module on intake kiosk. | [PS] |
| REQ-05 | "Multilingual, multi-accent voice capture in noisy hospital environments across Hindi, English, and major regional languages." | Voice pipeline must operate accurately in 60–80 dB ambient hospital noise across accents. | Noise cancellation DSP, directional microphone array handling, and fine-tuned Indian ASR (Bhashini/IndicASR). | [PS] |
| REQ-06 | "Accessibility for low-literacy and elderly users through intuitive icon-driven UI, audio prompts, and conversational guidance... zero training." | Kiosk cannot rely purely on text reading; must provide complete audio-visual duality. | Every UI element accompanied by TTS voiceover, high-contrast visual icons, and zero-text navigation pathways. | [PS] |
| REQ-07 | "SOCRATES framework for pain/symptoms... adaptive questioning dynamically branching based on chief complaint." | AI history engine must dynamically branch based on clinical logic (Onset, Character, Radiation, Associations, Time, Exacerbating, Severity). | Finite-state dialogue manager constrained by medical decision trees to prevent hallucinated diagnostic paths. | [PS] |
| REQ-08 | "Dual-mode input: every question answerable by speaking OR tapping." | Parallel interaction paradigm where touch inputs and voice inputs populate the same state variable. | UI reactive state bound simultaneously to ASR text intent extractor and touch button click handlers. | [PS] |
| REQ-09 | "Red-flag detection: AI flags emergency symptoms... triggers immediate priority alert to triage staff rather than routine queueing." | Real-time classification of life-threatening symptoms during intake. | Interrupt engine bypasses regular queue, emits audible/visual alert at triage nursing desk, and prints emergency token. | [PS] |
| REQ-10 | "OCR of handwritten and printed prescriptions... intelligent extraction of diagnoses, medications with dosages, investigation results, procedures." | Entity extraction must parse complex medical abbreviations, dosage formats, and lab reference ranges. | Named Entity Recognition (NER) model specialized in clinical entities (RxNorm, SNOMED CT, LOINC, NAMASTE). | [PS] |
| REQ-11 | "Abnormal-value highlighting and potential drug interactions." | Automated clinical decision support flagging lab outliers and contraindications. | Rule engine comparing parsed lab values against reference ranges and cross-checking extracted drug lists. | [PS] |
| REQ-12 | "DPDP Act 2023 and ABDM consent framework compliance... ABHA ID authentication... session data cleared immediately after submission." | Legal compliance regarding patient data rights, explicit consent, and data retention limits. | ABHA M1/M2/M3 OAuth flow, audio consent prompts, ephemeral memory storage purged upon session completion token issuance. | [PS] |
| REQ-13 | "Pushed to hospital HIS/EMR and linked to ABHA PHR via FHIR APIs... bilingual output (patient local audio, physician English/Hindi text)." | Interoperable data exchange with existing hospital information systems and national health stacks. | FHIR R4 Bundle mapping (Composition, OPConsultRecord, HealthDocumentRecord) sent via ABDM Gateway. | [PS] |
| Stakeholder | Primary Goal | Current Pain Point | System Need |
|---|---|---|---|
| OPD Patients (Rural, Elderly, Low-Literacy) | Obtain thorough clinical care without feeling rushed or misunderstood [INFERENCE]. | Cannot read complex digital forms; struggle to communicate history in 2 mins; carry messy paper files [PS]. | Conversational voice interface in local dialect, icon-driven touch fallback, audio-guided scanning, zero technical learning curve [PS]. |
| Ayurvedic Physicians (AIIA / AYUSH OPD) | Deliver personalized Ayurvedic care based on thorough Prakriti/Dashavidha Pariksha analysis [PS]. | Inability to perform 10-point Dashavidha Pariksha within 2-minute OPD slots, leading to forced history abbreviation [PS]. | Structured draft history summary presenting Prakriti/Agni/Koshtha traits and timeline before patient enters room [PS]. |
| Allopathic OPD Physicians | Quickly review complete patient background and focus on physical exam & clinical judgment [PS]. | Wasting 80% of consultation time sifting through paper records and asking repetitive baseline questions [PS]. | Concise 15-second scannable clinical summary with highlighted red-flags, abnormal lab values, and drug histories [PS]. |
| Triage / Nursing Staff | Identify critical/emergency cases instantly amidst massive daily crowds (4,000–10,000/day) [PS]. | Overwhelmed by OPD crowds; risk missing acute chest pain, stroke, or severe dyspnea in routine waiting lines [PS]. | Automated red-flag emergency alert system routing high-risk patients directly to emergency bays [PS]. |
| Hospital IT & ABDM Administrators | Ensure digital record compliance and integration without crashing local HIS infrastructure [INFERENCE]. | Legacy HIS systems lack FHIR conversion capabilities; paper records create massive data siloing [PS]. | Standardized ABDM-compliant FHIR R4 API connectors with zero persistent local data caching on public terminals [PS]. |
| Feature / Subsystem | Direct Justification (Requirement ID) | Functional Target |
|---|---|---|
| Bhashini Multi-dialect Voice Pipeline | REQ-05, REQ-06 |
Converts spoken regional Indian speech into standardized text in noisy (>65 dB) environments. |
| Dual-Mode Touch/Voice UI Module | REQ-06, REQ-08 |
Enables low-literacy users to complete intake via voice, tap, or hybrid interaction. |
| SOCRATES Clinical Dialogue Engine | REQ-07 |
Automatically branches symptom probing (Onset, Location, Duration, Character, Aggravating, Relieving, Severity). |
| Dashavidha Pariksha Intake Module | REQ-02 |
Captures Ayurvedic constitutional & diagnostic parameters (Prakriti, Vikriti, Agni, Koshtha, Ahara-Vihara). |
| Emergency Red-Flag Classifier | REQ-09 |
Identifies life-threatening symptoms and issues real-time priority triage alerts to nursing staff. |
| Handwritten Prescription OCR Engine | REQ-03, REQ-10 |
Extracts drugs, dosages, diagnoses, and lab results from unstructured physical medical documents. |
| Timeline & Lab Outlier Highlighting | REQ-04, REQ-11 |
Chronologically orders parsed historical records and flags abnormal values and drug-drug interactions. |
| Bilingual Summary Generator | REQ-13 |
Delivers audio confirmation to patient in native tongue and formatted structured clinical text to physician in English/Hindi. |
| ABHA Consent & FHIR R4 Bundle Service | REQ-12, REQ-13 |
Authenticates ABHA ID, records explicit DPDP-compliant consent, and posts FHIR OPConsult/HealthDocument bundles. |
| Session Ephemeral Data Purge | REQ-12 |
Clears all voice recordings, local document scans, and PII from memory instantly upon session release. |
| Data Type | Purpose | Ideal Real-World Source | Hackathon Substitute | Access Method | Availability | Privacy Sensitivity | Confidence |
|---|---|---|---|---|---|---|---|
| Symptom Ontology & SOCRATES Trees | Guided clinical dialogue branching | SNOMED CT / BMJ Best Practice / Ayush Standard Terminologies | Open-source SNOMED CT subset + Custom JSON decision trees | API / Local Database | Publicly Available | Non-sensitive | [KNOWN] |
| AYUSH Diagnostic Taxonomies | Capturing Prakriti, Vikriti, Agni, Dashavidha Pariksha | NAMASTE Portal (National Ayush Morbidity Codes) [RESEARCHED] | Digitized NAMASTE CSV dataset & AIIA intake clinical protocols | Web Portal / CSV Export | Publicly Available | Non-sensitive | [RESEARCHED] |
| Handwritten Indian Prescriptions | Training & evaluating OCR engine | Hospital EMR historical physical scan archives | Synthetically generated handwritten medical notes & public Kaggle medical prescription datasets | File System / Dataset download | Restricted (Real) / Open (Synthetic) | High (Real) / None (Synthetic) | [RESEARCHED] |
| Lab Value Reference Ranges | Highlighting out-of-range diagnostic tests | LOINC Database / NABL Accredited Pathology Standards | Standard Pathology Reference Range Table (JSON) | Direct Ingestion | Publicly Available | Non-sensitive | [KNOWN] |
| Drug-Drug Interaction Rules | Flagging dangerous drug combinations | National Formulary of India / NLM RxNorm / PubChem APIs | Curated list of top 100 common OPD drug interaction pairs | Local SQLite DB | Publicly Available | Non-sensitive | [KNOWN] |
| Indian Speech Audio Corpora | Multi-accent speech recognition fine-tuning | AI4Bharat IndicSUPERB / Bhashini Datasets [RESEARCHED] | IndicSpeech public audio samples + simulated noise augmentation | Download / API | Publicly Available | Non-sensitive | [RESEARCHED] |
| ABDM FHIR R4 Structure Definitions | Generating compliant Health Records | NRCeS India FHIR Implementation Guide v6.5.0 [RESEARCHED] | Official NRCeS FHIR JSON Schema templates | Git Repository | Publicly Available | Non-sensitive | [RESEARCHED] |
+---------------------------------------------------------------------------------------------------+
| EXISTING SOLUTIONS vs MEDIKIOSK |
+------------------------------------+--------------------------------+-----------------------------+
| System Class | Primary Operational Focus | Unmet Critical Gap |
+------------------------------------+--------------------------------+-----------------------------+
| e-Hospital (NIC) / ORS.gov.in | Admin Registration & Queuing | Zero clinical intake / OCR |
| Commercial Check-in Kiosks | Admin & Billing (Private Ops) | No Indian languages/voice/AI|
| Tele-triage & Mobile Apps | Remote Teleconsultation | Requires high digital literacy|
| Generic Scanner / OCR Utilities | Image PDF Archiving | No medical entity extraction|
| MEDIKIOSK (PROPOSED) | Multimodal AI Intake & OCR | FULLY INTEGRATED SOLUTION |
+------------------------------------+--------------------------------+-----------------------------+
[KNOWN][PS][PS][INFERENCE]+----------------------------------------------------------------------------------------------------+
| CANDIDATE ARCHITECTURE COMPARISON |
+-------------------+--------------------------------+-----------------------+-----------------------+
| Dimension | Approach 1: Static Rule Kiosk | Approach 2: LLM Voice | Approach 3: Hybrid AI |
+-------------------+--------------------------------+-----------------------+-----------------------+
| Primary Engine | Form Decision Trees | Pure Cloud GenAI LLM | Constrained FSM + Edge|
| Input Method | Touch Only (No Voice) | Natural Speech Only | Multimodal Voice+Touch|
| Hardware Cost | Low ($300/terminal) | High (GPU / Cloud API)| Moderate ($600/term) |
| Offline Capability| 100% Offline | 0% (Requires Internet)| High (Local Fallback) |
| Clinical Risk | Zero (Fixed paths) | High (Hallucinations) | Low (Guardrailed LLM) |
| Low-Literacy Fit | Poor | Moderate | Excellent |
+-------------------+--------------------------------+-----------------------+-----------------------+
The Hybrid Multimodal Architecture (Approach 3) is selected [INFERENCE].
REQ-08), Indian language ASR (REQ-05), and low-literacy audio guidance (REQ-06) [PS].+---------------------------------------------------------------------------------------------------+
| MEDIKIOSK SYSTEM ARCHITECTURE |
+---------------------------------------------------------------------------------------------------+
| 1. PRESENTATION LAYER (Kiosk Terminal & Physician Dashboard) |
| +-------------------------------------+ +----------------------------------------------------+ |
| | Patient Kiosk UI (React/Electron) | | Physician Web Portal (React/Tailwind) | |
| | - Audio-Visual Dual UI | | - 15-sec Scannable Summary Dashboard | |
| | - Push-To-Talk / Beamformed Mic | | - Red-Flag Alert Popups & Timeline View | |
| | - Document Feed Scanner / Camera | | - Editable Draft Summary / Acceptance Control | |
| +-------------------------------------+ +----------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------+
| 2. EDGE INTERACTION & STATE ENGINE (Kiosk Local Gateway - Node.js/Python) |
| +---------------------------------------------------------------------------------------------+ |
| | - Finite State Machine (FSM) Dialogue Controller (SOCRATES & Dashavidha Engine) | |
| | - Local Audio Noise Suppression & Voice Activity Detection (VAD - WebRTC) | |
| | - Session Ephemeral Cache (In-Memory SQLite - Purged post-submission) | |
| +---------------------------------------------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------+
| 3. AI & INTELLIGENCE PIPELINE LAYER (Cloud / On-Prem Application Server) |
| +--------------------------+ +--------------------------+ +----------------------------------+ |
| | ASR / TTS Engine | | Medical Document AI | | Summary & Red-Flag Engine | |
| | - Bhashini Streaming API | | - EasyOCR / PaddleOCR | | - Llama-3-8B / Med-Gemma LLM | |
| | - AI4Bharat IndicConformer| | - TrOCR (Handwritten) | | - Red-Flag Rule Engine (Regex) | |
| | - IndicTTS Audio Prompts | | - NER Entity Extractor | | - Drug-Interaction Matrix | |
| +--------------------------+ +--------------------------+ +----------------------------------+ |
+---------------------------------------------------------------------------------------------------+
| 4. INTEROPERABILITY, CONSENT & SECURITY LAYER |
| +---------------------------------------------------------------------------------------------+ |
| | - ABDM Gateway Connector (ABHA M1/M2/M3 Verification & OAuth 2.0) | |
| | - NRCeS FHIR R4 Bundle Mapper (OPConsultRecord, HealthDocumentRecord, Condition) | |
| | - DPDP Act 2023 Consent Manager (Granular Consent Logs & Audio Disclaimers) | |
| | - Crypto Engine (AES-256 Payload Encryption, TLS 1.3 Transit Protection) | |
| +---------------------------------------------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------+
| 5. DATA & STORAGE LAYER |
| +-----------------------------------+ +------------------------------------------------------+ |
| | Central HIS Database (PostgreSQL) | | FHIR Data Repository (HAPI FHIR Server) | |
| +-----------------------------------+ +------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------+
| Layer | Selected Technology | Why Chosen | Alternative Considered | Why Alternative Rejected |
|---|---|---|---|---|
| Kiosk Frontend | React.js + Electron (TailwindCSS) | Cross-platform desktop performance on kiosk hardware; seamless touch event handling and hardware device access [INFERENCE]. | Flutter Desktop | Slower plugin ecosystem for specialized document scanner peripherals [INFERENCE]. |
| Kiosk Edge Backend | Node.js (Express) + Python FastAPI | Node.js handles real-time WebSockets & UI state; Python manages local ML/OCR execution [INFERENCE]. | Pure Java Spring Boot | Heavy memory footprint on low-cost kiosk mini-PCs [INFERENCE]. |
| ASR (Speech-to-Text) | Bhashini / AI4Bharat IndicConformer API | Superior accuracy for Indian regional accents and 22 scheduled languages [RESEARCHED]. | OpenAI Whisper API | Lower performance on regional Indian dialects and non-standard accents [RESEARCHED]. |
| TTS (Text-to-Speech) | IndicTTS (AI4Bharat) / Bhashini TTS | Natural Indian language vocalization with multi-gender audio prompt capability [RESEARCHED]. | Google Cloud TTS | High recurring API cost for high-throughput public hospital queues [INFERENCE]. |
| Document OCR | PaddleOCR + TrOCR (Transformer OCR) | PaddleOCR excels at printed tables/reports; TrOCR handles handwritten Indian scripts [RESEARCHED]. | Tesseract OCR | Extremely poor accuracy on handwritten clinical notes [INFERENCE]. |
| Clinical NLP / Summarizer | Med-Gemma-2B / Llama-3-8B-Instruct (Quantized) | Compact model runnable on edge or local hospital server; zero data leakage to public clouds [INFERENCE]. | OpenAI GPT-4o | Regulatory and privacy concerns regarding sending patient PII to overseas cloud [PS]. |
| Database | PostgreSQL + pgvector | Robust relational storage for HIS records with vector support for medical ontology retrieval [INFERENCE]. | MongoDB | Weaker transaction safety for critical healthcare billing/consult records [INFERENCE]. |
| ABDM / FHIR Mapping | HAPI FHIR Library (Java/Python) | Official open-source standard for NRCeS India FHIR R4 Bundle validation [RESEARCHED]. | Custom JSON Builder | High risk of emitting non-compliant FHIR bundles rejected by ABDM gateway [RESEARCHED]. |
| Hardware Terminal | 15.6" Industrial Touchscreen + Push-to-Talk Mic + Document Camera | Durable, washdown-capable IP65 glass; directional microphone mitigates ambient OPD noise [INFERENCE]. | Standard Consumer iPad / Tablet | Fragile glass, poor directional audio capture, lack of integrated document feeder [INFERENCE]. |
+----------------------------------------------------------------------------------------------------+
| AI/ML COMPONENT MATRIX |
+--------------------------+-----------------------+------------------------+------------------------+
| Component | Primary Input | Generated Output | Failure Mode & Oversight|
+--------------------------+-----------------------+------------------------+------------------------+
| Bhashini ASR Engine | Spoken Audio Stream | Raw Transcript Text | Low SNR -> Fallback to |
| | (Hindi/Regional) | | Touchscreen selection |
+--------------------------+-----------------------+------------------------+------------------------+
| SOCRATES Dialogue FSM | Extracted Intents | Next Question / Prompt | Off-topic response -> |
| | | State | Re-prompt via audio |
+--------------------------+-----------------------+------------------------+------------------------+
| Document OCR & NER | Scanned Prescriptions/| Structured Entities | Unclear handwriting -> |
| | Lab Images | (Medication, Dosages) | Flagged for Rx verify |
+--------------------------+-----------------------+------------------------+------------------------+
| Clinical Summarizer | Conversation Logs + | Structured Draft HPI | Physician manual edit |
| | Digitized Entities | Summary | & accept/reject toggle |
+--------------------------+-----------------------+------------------------+------------------------+
-- Ephemeral Kiosk Session Table (Cleared post-submission)
CREATE TABLE kiosk_sessions (
session_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
kiosk_id VARCHAR(50) NOT NULL,
abha_number VARCHAR(17),
selected_language VARCHAR(10) DEFAULT 'hi',
consent_granted BOOLEAN NOT NULL DEFAULT FALSE,
consent_timestamp TIMESTAMP WITH TIME ZONE,
current_state VARCHAR(50) NOT NULL,
is_red_flag BOOLEAN DEFAULT FALSE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Clinical History Intake Responses
CREATE TABLE clinical_responses (
response_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
session_id UUID REFERENCES kiosk_sessions(session_id) ON DELETE CASCADE,
question_key VARCHAR(100) NOT NULL, -- e.g., 'CHIEF_COMPLAINT', 'SOCRATES_ONSET'
raw_audio_path VARCHAR(255),
transcript_text TEXT,
structured_value JSONB, -- Mapped SNOMED / NAMASTE code
input_mode VARCHAR(10) CHECK (input_mode IN ('VOICE', 'TOUCH', 'HYBRID'))
);
-- Extracted Document Intelligence Records
CREATE TABLE digitized_documents (
doc_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
session_id UUID REFERENCES kiosk_sessions(session_id) ON DELETE CASCADE,
document_type VARCHAR(50), -- 'PRESCRIPTION', 'LAB_REPORT', 'DISCHARGE_SUMMARY'
doc_date DATE,
raw_ocr_text TEXT,
extracted_entities JSONB, -- {medications: [], diagnoses: [], lab_results: []}
abnormal_flags JSONB
);
-- Formatted Physician Summary (Pushed to HIS)
CREATE TABLE clinical_summaries (
summary_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
session_id UUID REFERENCES kiosk_sessions(session_id),
patient_abha VARCHAR(17) NOT NULL,
chief_complaint TEXT NOT NULL,
hpi_structured JSONB NOT NULL,
ayush_dashavidha JSONB, -- Dashavidha Pariksha object
past_history JSONB,
timeline_summary JSONB,
fhir_bundle_json JSONB NOT NULL,
physician_status VARCHAR(20) DEFAULT 'DRAFT' -- 'DRAFT', 'ACCEPTED', 'AMENDED'
);
[ Kiosk Terminal ]
|
+---> Internet Active? ---> YES ---> [ Cloud Bhashini ASR / ABDM Gateway ]
|
+---> NO (Network Down)
|
v
[ Edge Fallback Protocol ]
1. Switch Audio to Local Pre-recorded Audio Prompt Assets (.wav)
2. Force Touchscreen Icon Selection Mode
3. Store Intake JSON in Local SQLite / IndexedDB Storage
4. Generate Local Queue Token with Offline Flag
5. Background Sync Daemon retries every 30s to push pending FHIR Bundles to HIS
+----------------------------------------------------------------------------------------------------+
| HACKATHON MVP BUILD SCOPE |
+----------------------------------------------------------------------------------------------------+
| MUST BUILD (Core Target) |
| - Multimodal React Kiosk UI (Hindi & English Voice + Touch) |
| - Bhashini Speech API Integration for ASR & Audio Prompts |
| - SOCRATES Decision-Tree Engine for Chief Complaint Probing |
| - AYUSH Dashavidha Pariksha Intake Module (Prakriti/Agni/Koshtha) |
| - OCR Pipeline for Printed & Handwritten Prescriptions (PaddleOCR/TrOCR) |
| - Real-time Emergency Red-Flag Triage Alert Pop-up System |
| - Physician Summary Dashboard (15-second scannable format with edit/accept controls) |
| - NRCeS FHIR R4 OPConsultRecord Bundle JSON Generator |
+----------------------------------------------------------------------------------------------------+
| SHOULD BUILD (High Priority Stretch) |
| - Local SQLite offline intake fallback mode |
| - Lab report out-of-range value auto-highlighting |
| - Mock ABHA OTP Authentication flow |
+----------------------------------------------------------------------------------------------------+
| NICE TO HAVE (Post-Hackathon Roadmap) |
| - Full 22-language translation support |
| - Multi-camera hardware scanner array controls |
| - Full HL7 FHIR HAPI Server deployment |
+----------------------------------------------------------------------------------------------------+
| DO NOT BUILD DURING HACKATHON |
| - Physical hardware kiosk shell or custom metal fabrication |
| - Direct billing/payment gateway integrations |
| - Native HIS database rewrites |
+----------------------------------------------------------------------------------------------------+
To present a realistic end-to-end demonstration without relying on live government production endpoints or physical hardware kiosks:
+----------------------------------------------------------------------------------------------------+
| LIVE DEMO SIMULATION ARCHITECTURE |
+----------------------------------------------------------------------------------------------------+
| Component | Real-World Endpoint | Demo Simulation Strategy |
+-------------------------+---------------------------------+----------------------------------------+
| Patient Terminal | Custom 15" Metal Kiosk | Laptop Touchscreen + Push-To-Talk Mic |
| ABDM Gateway | Live NHA ABDM Production API | Sandbox Mock ABDM Server with Valid ABHA|
| Document Ingestion | Hardware Sheetfeed Scanner | Pre-loaded Prescription Scans & Camera |
| Hospital HIS | AIIA Enterprise EMR | Custom Web Physician Dashboard |
| Hospital Noise | Busy OPD Waiting Hall | Background OPD Ambient Noise Audio Track|
+-------------------------+---------------------------------+----------------------------------------+
Target Deadline: 30 September 2026 [PS]
+---------------------------------------------------------------------------------------------------+
| IMPLEMENTATION ROADMAP (2026) |
+---------------------------------------------------------------------------------------------------+
| Phase 1: Foundation & Data Architecture (Weeks 1-2 | Target: July 15, 2026) |
| - Finalize FSM clinical decision trees (SOCRATES & Dashavidha Pariksha) |
| - Setup Bhashini/AI4Bharat API keys and local speech pipeline testbed |
| - Design PostgreSQL schema & NRCeS FHIR R4 JSON templates |
+---------------------------------------------------------------------------------------------------+
| Phase 2: Core Engine & OCR Development (Weeks 3-5 | Target: August 15, 2026) |
| - Implement React Audio-Visual Kiosk UI with dual voice/touch state management |
| - Train/Fine-tune TrOCR pipeline on sample handwritten Indian prescriptions |
| - Build Red-Flag triage classifier & emergency notification service |
+---------------------------------------------------------------------------------------------------+
| Phase 3: Integration & ABDM Pipeline (Weeks 6-7 | Target: September 5, 2026) |
| - Integrate Document Intelligence OCR with Summary Synthesizer |
| - Connect ABHA M1/M2 sandbox authentication and FHIR bundle generator |
| - Develop Physician Consultation Web Dashboard |
+---------------------------------------------------------------------------------------------------+
| Phase 4: Field Testing, Tuning & Polish (Weeks 8-9 | Target: September 20, 2026) |
| - Conduct acoustic noise stress testing (simulated 75 dB OPD background noise) |
| - Usability testing with non-tech-savvy users; refine audio prompt timing |
| - Final demo freeze, documentation, and presentation rehearsals |
+---------------------------------------------------------------------------------------------------+
| Test Category | Target Metric / Evaluation Method | Test Dataset / Tool | Success Threshold |
|---|---|---|---|
| ASR Voice Accuracy | Word Error Rate (WER) under simulated 70 dB hospital noise. | 100 sample audio recordings in Hindi/Regional dialects with background chatter. | WER < 15% [INFERENCE] |
| OCR Entity Extraction | Precision & Recall on handwritten medical notes. | 50 real/synthetic scanned Indian handwritten prescriptions. | Entity Recall > 85% [INFERENCE] |
| Red-Flag Sensitivity | False Negative Rate on acute life-threatening symptoms. | 30 test scenarios (15 emergency, 15 non-emergency). | 0% False Negatives (Zero missed emergencies) [INFERENCE] |
| Intake Speed | Time required for patient to complete full case history. | Usability trials with 10 simulated elderly/rural test users. | Intake completion < 3.5 minutes [INFERENCE] |
| Physician Review Efficiency | Time spent by physician reading intake summary. | Stopwatch measurement across 20 mock clinical encounters. | Summary review < 15 seconds [INFERENCE] |
| FHIR Compliance | Conformance to NRCeS FHIR R4 Implementation Guide. | Official NRCeS FHIR Validator Tool. | 100% schema validation pass (0 errors) [RESEARCHED] |
| DPDP Compliance | Verification of session memory purge post-submission. | Automated memory dump inspection scripts. | Zero persistent PII found on kiosk terminal [INFERENCE] |
| Risk Event | Likelihood | Impact | Severity | Mitigation Strategy |
|---|---|---|---|---|
| Acoustic Failure in High-Noise OPDs | High | High | HIGH | Hardware directional noise-cancelling microphone array + automatic UI fallback to touchscreen icons if WER spikes [INFERENCE]. |
| Unreadable Handwritten OCR Scans | High | Medium | MEDIUM | Display original cropped image scan side-by-side on physician screen with low-confidence visual warnings [INFERENCE]. |
| Network Outage Disruption | Medium | High | HIGH | Local SQLite edge caching; offline touch intake mode generating offline queue tokens for background HIS sync [INFERENCE]. |
| Patient Hallucination/Confusion | Medium | Medium | MEDIUM | Maximize visual icon clarity, provide clear audio prompts, and limit open-ended voice prompts to structured SOCRATES categories [INFERENCE]. |
| ABDM API Latency / Downtime | Medium | Medium | MEDIUM | Async queuing for ABDM FHIR pushes; local queue generation guarantees OPD flow is never blocked by external gateways [INFERENCE]. |
+----------------------------------------------------------------------------------------------------+
| KEY SYSTEM DIFFERENTIATORS |
+--------------------------+------------------------------------+------------------------------------+
| Feature Dimension | Generic Hackathon Entry | MEDIKIOSK PLATFORM |
+--------------------------+------------------------------------+------------------------------------+
| Clinical Intake Framework| Generic English Chatbot | SOCRATES + AYUSH Dashavidha System |
| User Accessibility | Text-heavy Mobile App | Audio-Visual Duality for Low-Literacy|
| Document Handling | Basic PDF Storage | Handwritten Medical OCR & Timeline |
| Emergency Management | Sequential Queueing | Real-Time Triage Red-Flag Engine |
| Standards Compliance | Custom Proprietary JSON | Native NRCeS ABDM FHIR R4 Bundles |
+--------------------------+------------------------------------+------------------------------------+
[ASSUMPTION][NEEDS VALIDATION]Proceed with building MediKiosk, a hybrid multimodal clinical history and document intelligence platform. It uniquely solves the 2-minute OPD bottleneck in Indian hospitals by combining Bhashini voice AI, SOCRATES/Dashavidha structured intake, handwritten prescription OCR, and NRCeS ABDM FHIR compliance [PS].
React.js + Electron (Frontend UI), Node.js / Python FastAPI (Edge Server), Bhashini / IndicConformer (ASR/TTS), PaddleOCR + TrOCR (Document AI), Med-Gemma-2B (Summarizer), PostgreSQL (Database), HAPI FHIR (ABDM Mapper).
Bhashini Streaming ASR API stability and latency under regional Indian accents [RESEARCHED].
Acoustic noise interference (60–80 dB) in crowded public hospital OPD queues degrading speech recognition accuracy [INFERENCE].
Accurate OCR and clinical entity extraction from unstructured, handwritten Indian physician prescriptions [INFERENCE].
https://drive.google.com/file/d/1mQ6Qp2MKL8JXdL2kJYqV-SFqcfbxSvrd/view?usp=drive_link).Test Bhashini ASR accuracy against sample regional voice audio recorded with simulated 70 dB background OPD noise.
88 / 100 (Highly feasible for a 4–6 student team using available open-source AI models and standard web frameworks) [INFERENCE].
High (Supported by detailed problem statement specifications, verified BMJ study metrics, official ABDM FHIR standards, and established Bhashini APIs) [PS], [RESEARCHED].
BMJ Open (2017) Study on Primary Care Consultation Time
https://bmjopen.bmj.com/content/7/10/e017902 [RESEARCHED]National Health Authority (NHA) & NRCeS FHIR Implementation Guide
https://www.nrces.in/ndhm/fhir/r4/index.html [RESEARCHED]Digital India Bhashini Mission & AI4Bharat
https://www.bhashini.ai/docs [RESEARCHED]NAMASTE Portal - Ministry of Ayush
https://elms.ayush.gov.in/ayushterminology [RESEARCHED]What we think about this platform, and how it can help people solve SIH problems.
" 30M+ Google Gemini tokens went into researching, analyzing, and understanding all 233 SIH problem statements to make them easier to explore and solve. "
" The platform makes SIH problem statements much easier to understand. Having everything from the problem details to the procedure in one place. "
" Oh, you got ads? Well, we need to make money too! But, you can use our platform to help you win SIH and take home the prize money. "
Get in touch if you need help bringing your ideas to life, or simply have an interesting idea you'd like to talk through. I'm always open to a conversation and exploring new possibilities together.