Sharpen AI

The Goal

Sharpen AI is a browser extension that transforms real-time video conferences into highly structured, actionable summaries.

The primary objective was to build a completely friction-free, non-invasive meeting assistant. While most enterprise meeting utilities rely on external recording bots that physically join calls as separate participants—creating a disruptive environment for attendees—Sharpen AI runs entirely within the local browser runtime. This allows it to capture data organically without interrupting the meeting ecosystem.

The target was a seamless, automated user experience: after a single installation, the system automatically detects active conference sessions, manages stream data quietly in the background, and dispatches polished summaries directly to workspace participants post-meeting.

Phase 1: Native Data Integration

The initial architecture focused on establishing a zero-footprint presence by capturing real-time textual data streams generated natively during live video calls. Leveraging pre-existing session metadata felt like an elegant way to avoid the heavy computational overhead of server-side processing.

However, extracting this data continuously without degrading the user interface presented an immediate challenge. To ensure a clean workspace for the user, I developed a custom script injection framework that:

• Programmatically initialized the native interface data streams upon call entry.
• Abstracted and decoupled the visual data rendering layers from the active viewport.
• Injected a customized control interface, allowing users to toggle local visibility on-demand without interrupting the background data collection loop.

The Architectural Evolution: Dual-Stream Context Synthesis

While the early text-capture framework worked smoothly, relying solely on single-source platform text interfaces introduced a significant bottleneck: real-time transcription quality fluctuates heavily across diverse dialects, multi-language transitions, and platform updates.

To achieve production-grade accuracy, we pivoted to an audio-supported framework. However, standard audio streams introduce a classic engineering problem: Speaker Diarization (identifying precisely who spoke which words). Traditional acoustic models frequently struggle to accurately resolve individual names from low-fidelity web call feeds.

To solve this, I engineered a proprietary Dual-Stream Context Synthesis Engine utilizing advanced, long-context multimodal models.

Instead of processing text and audio through separate, isolated pipelines, the engine synthesizes both streams simultaneously. The system uses the real-time textual metadata as a structural identity map and temporal anchor, while utilizing the high-fidelity audio stream to definitively reconstruct the precise vocabulary spoken. By cross-referencing these two data streams, the platform achieves absolute speaker accuracy and pristine textual clarity, entirely bypassing the limitations of traditional acoustic diarization.

Engineering a Fault-Tolerant Streaming Pipeline

Capturing system-level audio under modern, highly sandboxed browser security configurations presents severe architectural limits. Because background service workers cannot maintain continuous media recorders directly, I orchestrated a decoupled, transient document pipeline to manage the active streams.

To protect user data against unexpected tab closures, network drops, or browser crashes, I developed a highly resilient client-side buffering architecture:

Enterprise Scaling & Architecture Realization

To support commercial viability, I engineered a robust enterprise licensing and seat-management architecture integrated natively with secure payment gateways. The backend bypasses simple consumer tier limitations, allowing corporate administrators to self-assign, revoke, and redistribute user seats dynamically across large organizations via a central dashboard.

The complete system infrastructure was successfully realized using a modern, decoupled stack:

• Frontend Architecture: Next.js paired with Tailwind CSS for high-performance extension interfaces and administrative portals.
• State & Identity Management: Cloud database infrastructure optimized to map nested organizational roles, active enterprise license compliance, and historical meeting metadata.
• Security & Gateway Infrastructure: Serverless cloud functions protected by signed webhook validation layers to ensure secure, real-time access provisioning.

Key Takeaways

Sharpen AI represents a complete exercise in secure browser systems engineering and product optimization. Navigating strict sandboxing constraints, building highly resilient client-side storage recovery mechanisms, and orchestrating sophisticated contextual data synthesis proved how platform limitations can be leveraged to engineer an immensely stable, enterprise-ready utility.