Connected-Device IoT Telemetry Pipeline

BUSINESS PROBLEM

A major medical equipment manufacturer based in Oregon ships a connected line of professional equipment that streams device telemetry through Azure IoT Hub. They had raw JSON events landing in cloud storage but no way to turn that firehose into something the service, engineering, or customer-experience teams could actually use. They engaged HERA to achieve:

· Near-real-time visibility into device state (which alerts are active right now, on which unit, at which customer site)

· A historical record of every property change (position state, motor state, alert active/cleared, firmware version) so service teams can spot recurring issues before a customer calls

· A foundation that downstream BI and dashboard tools can build on without re-parsing JSON every time

OUR APPROACH

After speaking with key stakeholders, end users, and other impacted parties, HERA utilized our skills in Snowflake (SQL, Streams, Tasks, stored procs in Python), Coalesce.io, Azure IoT Hub, JSON / semi-structured data, Git (Azure DevOps), Python, PowerShell to deliver end-to-end streaming + historical pipeline to deliver:

· Ingestion → curated raw → flattened stage → SCD2 interval table → consumption views. ~1-minute end-to-end latency from device emit to queryable Snowflake row.

· Dual-subgraph architecture so customer-facing streaming keeps running while historical data is replayed in parallel — eliminating the multi-day customer-impacting outage that backfills used to require.

· Version-aware JSON transforms that handle three concurrent firmware schema generations on a mixed device fleet without breaking older devices.

· Snowflake-Task-driven orchestration layer built to replace the default scheduler after a concurrency bug caused data corruption — moved scheduling into a

Snowflake stored procedure that calls the transform tool's REST API with proper overlap protection.

· Multiple production output views and operational metric views consumed by the data team and an operations dashboard.

· Operational runbooks for stale-stream recovery, historical backloads, and platform continuity ahead of my rotation off the engagement.

Project/Change Management

Requirements Gathering

SERVICES PERFORMED

API/Reporting Development

Technology Assessment

Custom Integration

VALUE DELIVERED

· Live in production since Feb 5, 2026. Currently storing ~742 million interval rows across ~171 GB of telemetry, growing every minute.

· ~1-minute average pipeline latency, with 98%+ of events queryable within 5 minutes of device emit.

· 97.5% reduction in data-quality defects after a major fix (multi-open interval bug): 798 corrupted records reduced to 20, all of which are explainable known device-clock issues.

· Zero customer-visible downtime during a full historical reload of 22 months of data — the dual-subgraph design I delivered means backfills now run alongside streaming instead of pausing it.

· 6 production deploys over the past three months, each shipped after staged dev/QA validation with no rollbacks required.

· Documentation handed off to the client's internal data team (operations runbook, deploy playbook, continuity plan) so the platform is sustainable long-term.

97.5%

Reduction in Data Quality Defects

Production Deploys

Customer-Visible Downtime