klinik-gunung-docs
AI

Gemini AI Integration for Health Screening

Comprehensive documentation of Google Gemini AI integration for automated KTP, Passport analysis, and document processing in the Klinik Gunung Semeru healthcare system

Gemini AI Integration for Health Screening

This comprehensive guide covers the integration of Google Gemini AI (formerly Bard) for automated document analysis in the Klinik Gunung Semeru healthcare management system. The AI system provides intelligent OCR (Optical Character Recognition) capabilities for Indonesian ID cards (KTP) and passports.

System Architecture

Overview

The AI integration consists of a dedicated Python microservice that communicates with Google Gemini AI to analyze identity documents and convert image formats. This service is completely separate from the main Laravel backend for scalability and performance.

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Frontend      │───▶│  Python AI       │───▶│   Gemini AI     │
│   (Next.js)     │    │  Microservice    │    │   (Google)      │
│                 │    │                  │    │                 │
│ • File Upload   │    │ • Image Analysis │    │ • OCR           │
│ • Camera        │    │ • Format         │    │ • Data          │
│ • Real-time     │    │ • Conversion     │    │ • Extraction    │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Technology Stack

Core Components

  • Google Gemini 2.0 Flash Lite: Primary AI model for document analysis
  • FastAPI: High-performance Python web framework
  • Pillow: Image processing and format conversion
  • Pillow-HEIF: HEIC/HEIF image format support
  • Redis: Caching and session management (optional)

Key Dependencies

fastapi==0.104.1
uvicorn[standard]==0.24.0
google-generativeai==0.3.2
pillow==10.1.0
pillow-heif==0.13.1
pydantic==2.5.0
pydantic-settings==2.1.0
redis==5.0.1
python-dotenv==1.0.0
aiofiles==23.2.1

AI Service Architecture

Service Structure

python/
├── app/
│   ├── main.py                 # FastAPI application entry point
│   ├── config.py               # Configuration management
│   ├── api/
│   │   └── v1/
│   │       ├── ktp_routes.py   # KTP analysis endpoints
│   │       └── passport_routes.py # Passport analysis endpoints
│   └── services/
│       ├── ktp_analyzer.py     # KTP AI analysis logic
│       ├── passport_analyzer.py # Passport AI analysis logic
│       └── heic_converter.py   # Image format conversion
├── requirements.txt            # Python dependencies
├── .env.example               # Environment variables template
└── README.md                  # Service documentation

Service Configuration

Environment Variables

# Gemini AI API Keys (Multiple for redundancy)
GEMINI_API_KEY_1=your_first_api_key
GEMINI_API_KEY_2=your_second_api_key
GEMINI_API_KEY_3=your_third_api_key
GEMINI_API_KEY_4=your_fourth_api_key
GEMINI_API_KEY_5=your_fifth_api_key

# Upload Configuration
UPLOAD_DIR=uploads/ktp
MAX_FILE_SIZE=52428800  # 50MB
ALLOWED_EXTENSIONS=jpg,jpeg,png,webp,heic,heif

# Redis (Optional)
REDIS_HOST=localhost
REDIS_PORT=6379

# CORS Origins
ALLOWED_ORIGINS=http://localhost:3000,http://localhost:3001

Configuration Class

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    # Gemini API Keys (up to 5 for redundancy)
    GEMINI_API_KEY_1: Optional[str] = None
    GEMINI_API_KEY_2: Optional[str] = None
    GEMINI_API_KEY_3: Optional[str] = None
    GEMINI_API_KEY_4: Optional[str] = None
    GEMINI_API_KEY_5: Optional[str] = None

    # Current key rotation index
    CURRENT_KEY_INDEX: int = 0

    # File upload settings
    UPLOAD_DIR: str = "uploads/ktp"
    MAX_FILE_SIZE: int = 50 * 1024 * 1024  # 50MB
    ALLOWED_EXTENSIONS: List[str] = [
        "jpg", "jpeg", "png", "webp", "heic", "heif"
    ]

    class Config:
        env_file = ".env"

Gemini AI Integration

AI Model Selection

  • Primary Model: gemini-2.0-flash-lite
  • Reason: Fast inference, cost-effective, good OCR capabilities
  • Fallback: Automatic key rotation for high availability

API Key Management

Multi-Key Redundancy System

The system supports up to 5 Gemini API keys for redundancy and load distribution:

@property
def GEMINI_API_KEYS(self) -> List[str]:
    """Combine all API keys into a list"""
    return [
        key for key in [
            self.GEMINI_API_KEY_1,
            self.GEMINI_API_KEY_2,
            self.GEMINI_API_KEY_3,
            self.GEMINI_API_KEY_4,
            self.GEMINI_API_KEY_5,
        ] if key  # Only include non-None keys
    ]

Automatic Key Rotation

def _is_quota_or_auth_error(self, exc: Exception) -> bool:
    """Detect quota/rate-limit/auth errors for key rotation"""
    msg = str(exc).lower()

    # Check common error patterns
    if "429" in msg or "rate limit" in msg or "quota" in msg:
        return True
    if "invalid" in msg and "key" in msg:
        return True
    if "permission" in msg or "forbidden" in msg:
        return True

    return False

Exponential Backoff

# Small exponential backoff before retrying
await asyncio.sleep(min(base_backoff * (2 ** (tried - 1)), 4.0))

Document Analysis Capabilities

Indonesian KTP Analysis

Input: KTP image (various formats supported) Output: Structured JSON with extracted information

Supported Fields:

{
  "NIK": "string (16 digits)",
  "Nama": "string",
  "Tempat Lahir": "string",
  "Tanggal Lahir": "string (DD-MM-YYYY)",
  "Jenis Kelamin": "LAKI-LAKI or PEREMPUAN",
  "Alamat": "string",
  "RT/RW": "string",
  "Kelurahan/Desa": "string",
  "Kecamatan": "string",
  "Agama": "string",
  "Status Perkawinan": "string",
  "Pekerjaan": "string",
  "Kewarganegaraan": "string",
  "Berlaku Hingga": "string",
  "Golongan Darah": "string or '-'"
}

AI Prompt for KTP Analysis:

base_prompt = """
Analisis gambar KTP Indonesia ini dan ekstrak informasi berikut dalam format JSON.
Pastikan setiap field diisi dengan akurat sesuai yang tertera di KTP.

Format JSON yang diharapkan:
{
    "NIK": "string",
    "Nama": "string",
    "Tempat Lahir": "string",
    "Tanggal Lahir": "string (format: DD-MM-YYYY)",
    "Jenis Kelamin": "LAKI-LAKI atau PEREMPUAN",
    "Alamat": "string",
    "RT/RW": "string",
    "Kelurahan/Desa": "string",
    "Kecamatan": "string",
    "Agama": "string",
    "Status Perkawinan": "string",
    "Pekerjaan": "string",
    "Kewarganegaraan": "string",
    "Berlaku Hingga": "string",
    "Golongan Darah": "string (atau - jika tidak ada)"
}

Catatan penting:
- Jika field tidak terlihat jelas atau tidak ada, isi dengan string kosong ""
- Pastikan format tanggal konsisten: DD-MM-YYYY
- Jenis Kelamin harus LAKI-LAKI atau PEREMPUAN (huruf kapital)
- Golongan darah isi dengan - jika tidak ada
- Berikan hanya JSON tanpa penjelasan tambahan
"""

Passport Analysis

Supported Fields:

{
  "Passport Number": "string",
  "Full Name": "string",
  "Place of Birth": "string",
  "Date of Birth": "string (DD-MM-YYYY)",
  "Gender": "MALE or FEMALE",
  "Nationality": "string",
  "Date of Issue": "string (DD-MM-YYYY)",
  "Date of Expiry": "string (DD-MM-YYYY)",
  "Passport Type": "string",
  "Issuing Country": "string"
}

Image Processing Pipeline

HEIC/HEIF Conversion

class HeicConverter:
    def __init__(self, quality: int = 85, max_width: int = 1920, max_height: int = 1080):
        self.quality = quality
        self.max_width = max_width
        self.max_height = max_height

    async def convert_to_jpeg(self, heic_content: bytes) -> str:
        """Convert HEIC/HEIF to JPEG format"""
        # Open HEIC image with Pillow-HEIF
        image = Image.open(io.BytesIO(heic_content))

        # Convert to RGB (JPEG doesn't support transparency)
        if image.mode in ('RGBA', 'LA', 'P'):
            background = Image.new('RGB', image.size, (255, 255, 255))
            if image.mode == 'P':
                image = image.convert('RGBA')
            background.paste(image, mask=image.split()[-1] if image.mode == 'RGBA' else None)
            image = background
        elif image.mode != 'RGB':
            image = image.convert('RGB')

        # Resize if too large (mobile optimization)
        if image.width > self.max_width or image.height > self.max_height:
            image.thumbnail((self.max_width, self.max_height), Image.Resampling.LANCZOS)

        # Convert to base64 for AI processing
        jpeg_buffer = io.BytesIO()
        image.save(jpeg_buffer, format='JPEG', quality=self.quality, optimize=True)
        jpeg_base64 = base64.b64encode(jpeg_buffer.getvalue()).decode('utf-8')

        return jpeg_base64

Mobile Optimization

  • Automatic image resizing for mobile devices
  • Quality optimization (85% JPEG quality)
  • Maximum dimensions: 1920x1080 pixels
  • Format conversion for AI compatibility

API Endpoints

Base URL

http://localhost:8000/api/v1/

KTP Analysis Endpoints

POST /api/v1/convert-heic

Convert HEIC/HEIF images to JPEG format.

Request: Multipart form data with file Response:

{
  "success": true,
  "message": "File converted successfully",
  "data": {
    "jpeg_base64": "base64_encoded_jpeg",
    "original_filename": "image.heic",
    "converted_filename": "image.jpg"
  }
}

POST /api/v1/analyze-ktp

Analyze Indonesian KTP using Gemini AI.

Parameters:

  • file: Image file (UploadFile)
  • mobile_optimization: "true" or "false" (Form field)

Response:

{
  "success": true,
  "message": "KTP analyzed successfully",
  "data": {
    "NIK": "1234567890123456",
    "Nama": "JOHN DOE",
    "Tempat Lahir": "JAKARTA",
    "Tanggal Lahir": "15-01-1990",
    "Jenis Kelamin": "LAKI-LAKI",
    "Alamat": "JL. EXAMPLE NO. 123",
    "RT/RW": "001/002",
    "Kelurahan/Desa": "EXAMPLE VILLAGE",
    "Kecamatan": "EXAMPLE DISTRICT",
    "Agama": "ISLAM",
    "Status Perkawinan": "BELUM KAWIN",
    "Pekerjaan": "SWASTA",
    "Kewarganegaraan": "WNI",
    "Berlaku Hingga": "SEUMUR HIDUP",
    "Golongan Darah": "O"
  }
}

GET /api/v1/api-key-status

Check current API key rotation status.

Response:

{
  "current_key_index": 1,
  "total_keys": 5,
  "keys_available": 5
}

Passport Analysis Endpoints

POST /api/v1/analyze-passport

Analyze passport documents using Gemini AI.

Parameters: Same as KTP analysis Response: Structured passport data in JSON format

Frontend Integration

React Component Integration

KTP Upload Component

// components/screening/KTPupload.tsx
const handleFileAnalysis = async (file: File, isMobile: boolean) => {
  const formData = new FormData();
  formData.append('file', file);
  formData.append('mobile_optimization', isMobile.toString());

  const response = await apiClient.analyzeKTP(file, isMobile);

  if (response.success && response.data) {
    // Process extracted KTP data
    updatePatientData('nik', response.data.NIK);
    updatePatientData('name', response.data.Nama);
    updatePatientData('place_of_birth', response.data["Tempat Lahir"]);
    // ... other fields
  }
};

Camera Integration

// Real-time camera capture analysis
const handleCameraCapture = async (imageSrc: string) => {
  const response = await fetch(imageSrc);
  const blob = await response.blob();
  const file = new File([blob], 'camera_capture.jpg', { type: 'image/jpeg' });

  const isMobile = /iPhone|iPad|iPod|Android/i.test(navigator.userAgent);
  const result = await apiClient.analyzeKTP(file, isMobile);

  if (result.success && result.data) {
    handleKTPAnalysis(result.data, file);
    toast.success('KTP berhasil dianalisis!');
  }
};

API Client Implementation

TanStack Query Integration

// lib/api/client.ts
export const apiClient = {
  analyzeKTP: async (file: File, isMobile: boolean = false) => {
    const formData = new FormData();
    formData.append('file', file);
    formData.append('mobile_optimization', isMobile.toString());

    const response = await fetch(`${AI_SERVICE_URL}/api/v1/analyze-ktp`, {
      method: 'POST',
      body: formData,
    });

    if (!response.ok) {
      throw new Error(`AI service error: ${response.status}`);
    }

    return response.json();
  },

  convertHEIC: async (file: File) => {
    const formData = new FormData();
    formData.append('file', file);

    const response = await fetch(`${AI_SERVICE_URL}/api/v1/convert-heic`, {
      method: 'POST',
      body: formData,
    });

    return response.json();
  }
};

Performance Optimization

Image Processing Optimizations

  • Progressive JPEG: Quality 85% with optimization
  • Smart Resizing: Max dimensions 1920x1080 for mobile
  • Format Conversion: Automatic HEIC/HEIF to JPEG
  • Memory Management: Streaming for large files

AI Request Optimizations

  • Batch Processing: Multiple images per request when possible
  • Caching: Redis caching for frequent analyses
  • Async Processing: Non-blocking AI calls with asyncio
  • Error Recovery: Automatic retry with exponential backoff

Rate Limiting & Quota Management

  • Key Rotation: Automatic switching between API keys
  • Quota Monitoring: Real-time quota usage tracking
  • Fallback Handling: Graceful degradation when AI unavailable
  • Load Balancing: Distribute requests across multiple keys

Error Handling & Monitoring

Error Types & Responses

Quota/Ratelimit Errors (429)

{
  "detail": "All Gemini API keys exhausted or invalid. Please try again later.",
  "status_code": 429
}

Invalid File Format (400)

{
  "detail": "Unsupported file format. Allowed: jpg, jpeg, png, webp, heic, heif",
  "status_code": 400
}

File Too Large (413)

{
  "detail": "File too large (max 50MB)",
  "status_code": 413
}

AI Processing Errors (500)

{
  "detail": "Error analyzing KTP: AI service temporarily unavailable",
  "status_code": 500
}

Monitoring & Logging

Health Checks

# Service health check
curl http://localhost:8000/health

# API key status
curl http://localhost:8000/api/v1/api-key-status

Performance Metrics

  • Response time monitoring
  • Success/failure rates
  • API key usage statistics
  • Image processing throughput

Logging Strategy

# Structured logging for AI operations
import logging

logging.info(f"KTP Analysis - Key: {current_key_index}, Duration: {duration}s")
logging.warning(f"Key rotation triggered - Key {old_index} exhausted")
logging.error(f"AI processing failed - {error_message}")

Deployment & Scaling

Docker Containerization

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Production Deployment

# Using Docker Compose
version: '3.8'
services:
  ai-service:
    build: .
    ports:
      - "8000:8000"
    environment:
      - GEMINI_API_KEY_1=${GEMINI_API_KEY_1}
      - GEMINI_API_KEY_2=${GEMINI_API_KEY_2}
    volumes:
      - ./uploads:/app/uploads
    restart: unless-stopped

Load Balancing

upstream ai_service {
    server ai-service-1:8000;
    server ai-service-2:8000;
    server ai-service-3:8000;
}

server {
    listen 80;
    server_name ai.klinikgunung.com;

    location / {
        proxy_pass http://ai_service;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Security Considerations

API Key Protection

  • Environment variable storage
  • No hardcoded keys in source code
  • Regular key rotation
  • Access logging without exposing keys

Data Privacy

  • No persistent storage of uploaded images
  • Immediate processing and cleanup
  • HTTPS-only communication
  • GDPR compliance for personal data

Input Validation

  • File type verification
  • Size limits enforcement
  • Image format validation
  • Malware scanning integration

Cost Optimization

Gemini AI Pricing Strategy

  • Flash Lite Model: Cost-effective for document OCR
  • Key Rotation: Distribute usage across multiple keys
  • Caching: Avoid duplicate analyses
  • Batch Processing: Combine multiple requests when possible

Usage Monitoring

# Track API usage
usage_stats = {
    'total_requests': 0,
    'successful_analyses': 0,
    'failed_analyses': 0,
    'average_response_time': 0,
    'key_usage': {i: 0 for i in range(5)}
}

Future Enhancements

Advanced AI Features

  • Multi-language Support: Additional document types
  • Batch Processing: Analyze multiple documents simultaneously
  • Quality Validation: Automated confidence scoring
  • Template Matching: Custom document formats

Performance Improvements

  • GPU Acceleration: For image processing
  • Edge Computing: Process images closer to users
  • Model Fine-tuning: Custom-trained models for specific documents

Integration Extensions

  • Blockchain Verification: Document authenticity validation
  • Biometric Matching: Facial recognition integration
  • Real-time Processing: WebRTC for live document capture

This Gemini AI integration provides a robust, scalable solution for automated document analysis in healthcare settings. The system combines cutting-edge AI capabilities with production-ready features like error handling, monitoring, and security, ensuring reliable operation in clinical environments.

Server Migration with Automatic Backup to Local Server

Comprehensive guide for migrating Laravel applications to new VPS servers with automatic backup system to local server for maximum data safety and minimal downtime.

Database Documentation

Comprehensive documentation of the Klinik Gunung Semeru healthcare management system database structure, including all tables, relationships, and migration details.