Kallol Chakraborty
<- projects/
PERSONAL PROJECT · LOCAL AI

Chhanda — Local AI Gateway

100% offline, privacy-first, production-hardened AI & RAG ecosystem for Android — named after the Bengali poetic meter (ছন্দ), dedicated to the developer's mother. Solo-developed, 14,500+ LOC, 100% Kotlin, zero cloud dependency.

Kotlin 2.1.0 · 14,500+ LOC Android 8.0+ (API 26 → 34) Gemma 4 · LiteRT-LM MVVM + Clean Architecture Gemma for Good Hackathon
GITHUB REPO VIEW ON GITHUB

Architecture

Presentation Layer
DashboardScreen · ChatScreen · KnowledgeBaseScreen · ConfigScreen · LogsScreen · WelcomeScreen
ViewModel + Domain Layer
SystemViewModel (2,022 LOC) · ChatViewModel · SendMessageUseCase · IngestDocumentUseCase · ScrapeUrlUseCase
Inference
LiteRTLMEngine · LiteRTEmbeddingEngine (512-dim) · JNI
RAG Pipeline
AndroidMultimodalIngestor · LocalVectorStore · TextChunker · ContextManager
Gateway Server
ChhandaServer (Ktor-CIO) · ServerOrchestrator · ModelProvisioner
Persistence
Room DB (4 DAOs) · EncryptedSharedPreferences · DataStore
Android Services
ForegroundService · Quick-Settings Tile · DownloadWorker · IngestionWorker

Zero-Cloud Local Inference

  • Google LiteRT-LM engine (litertlm-android:0.11.0) with native JNI acceleration.
  • Models: Gemma 2B, 4B, 4n — 4-bit GGUF quantized; optional TurboQuant KV-cache compression for long-context VRAM relief.
  • Auto model discovery: scans Downloads/, app-private storage, and shared external dirs.
  • Configurable context window 102 → 32,768 tokens via Settings slider.
  • Token-by-token streaming with live tokens/sec counter and Thinking Mode (<thought> toggle).
  • Hardware-aware slider init: 2GB RAM → 1024, 4GB → 2048, 8GB+ → 4096 tokens.

Adaptive Multimodal RAG Pipeline

Ingestion

  • PDF, DOCX, DOC, XLSX, XLS, CSV, TSV, XML, HTML, MD, JSON + OCR via ML Kit.
  • Jsoup-based web scraping with depth-first link following (ScrapeUrlUseCase).
  • Paragraph-first text chunking (TextChunker).

Retrieval

  • On-device MediaPipe embeddings, 512-dim vectors.
  • Int8 quantization → 75% storage reduction vs Float32.
  • Min-Heap PriorityQueue similarity search — O(N log K).
  • Adaptive thresholds: 0.80 chat / 0.60 explicit / 0.50 follow-up; pronoun-aware query expansion.

Production-Grade AI Gateway

  • Ktor-CIO coroutine-based server (no Netty/JNI overhead); embedded Web UI served over HTTP.
  • OpenAI-compatible REST API — works with VS Code Continue, IDEs, and mobile clients.
  • mDNS (NSD) service registration + QR-code sharing of connection config.
  • Mandatory X-API-Key header on all endpoints (KeyStore-backed), Leaky Bucket throttling (1 req/s/IP), semaphore queue (max 2 inference tasks), 30-second device reaper.
  • SSH reverse tunnel via localhost.run for zero-config remote access; active VPN warning.
  • Serves up to 20 concurrent clients.

Safety & Privacy — Defense in Depth

  • Prompt injection: keyword blacklist + heuristic regex + defensive delimiters ([USER_INPUT_START/END]).
  • PII redaction: bidirectional input/output masking of emails, phones, SSNs, cards.
  • Content filtering: regex detection of violent, illegal, self-harm content.
  • Ephemeral API mode — API conversations never persisted to disk.
  • Key storage in EncryptedSharedPreferences backed by Android KeyStore TEE/SE.
  • Biometric lock + PrivacyGuard, SafetyGuardrails, SslCertManager.
  • HuggingFace read-only token stored in KeyStore for authenticated model downloads.

Hardware Resilience

  • ThermalStatusTracker auto-reduces context window under thermal stress; MemoryPressureMonitor guards OOM.
  • 2.5s flush delay between model switches; hot model swap with safe engine teardown.
  • Lifecycle-aware telemetry (12%+ battery savings); persistent inference via Wake/Wi-Fi locks.
  • System tile (ChhandaTileService) for instant server status; crash handler for zero-loss restarts.

Performance & Security Benchmarks

Metric Standard Chhanda (Hardened)
Vector storageFloat32 (4 bytes/dim)Int8 quantized — 75% less disk
Search complexityO(N log N) full sortO(N log K) min-heap — 3× faster top-K
Network securityOpen portLeaky Bucket + API key + device limit
BatteryActive background pollingLifecycle-aware — 12% less battery
Token storagePlain-text prefsEncryptedSharedPreferences (TEE)
ConcurrencyUnboundedSemaphore (max 2 tasks) — crash-free
Cold startEager initializationLazy DI (dagger.Lazy<>) — 15% faster

Codebase at a Glance

app/src/main/java/com/chhanda/ai/
├── data/
│   ├── inference/          ChhandaServer.kt (48KB) · LiteRTLMEngine · LiteRTEmbeddingEngine
│   │                       AndroidMultimodalIngestor (27KB) · ServerTemplateProvider (40KB)
│   └── repository/         AppDatabase (Room, 4 DAOs) · LocalVectorStore · ModelProvisioner
│                           HardwareMonitor · SettingsRepository · SecurityRepository
├── domain/
│   ├── model/              ContextManager · PersonaManager · TextChunker · RAGMetricsManager
│   └── usecase/            SendMessageUseCase (46KB) · ScrapeUrlUseCase · IngestDocumentUseCase
├── presentation/
│   ├── ui/                 DashboardScreen (88KB) · ConfigScreen (55KB) · KnowledgeBaseScreen
│   │                       ChatScreen · LogsScreen · WelcomeScreen · ChatComponents
│   └── viewmodel/          SystemViewModel (54KB) · ChatViewModel · ChatUiState
├── service/                ChhandaForegroundService · ChhandaTileService · DownloadWorker
│                           IngestionWorker · CloudSyncWorker · InferenceService
└── util/                   SafetyGuardrails · Localization (32KB) · VoiceAssistant · DocumentGenerator
                            PrivacyGuard · ThermalStatusTracker · CrashHandler

Shipped with MANUAL.md, HACKATHON_SUBMISSION.md, FAQSheet.md, TestCases.md, an Android instrumentation test suite (EngineInferenceTest, RAGPipelineTest) and unit tests (SystemViewModelTest, ContextManagerTest).

Tech Stack

Kotlin 2.1.0 (100% Kotlin) Android 8.0+ (API 26→34) LiteRT-LM 0.11.0 Gemma 2B / 4B / 4n (GGUF 4-bit) Ktor-CIO Room · DataStore · Hilt DI MediaPipe (embedding/OCR) Jsoup · Apache POI · ML Kit Android KeyStore TEE
↑↓ navigate ↵ open