CelerData Blog

Smarter Scaling in CelerData Cloud BYOC

Kiran Kamreddy — Tue, 03 Mar 2026 00:51:53 GMT

Real-time analytics has become the engine behind modern digital experiences, powering instant dashboards, interactive analytics, personalized recommendations, and operational decisioning.

February 2026 Highlights: Sub-Second Agentic Analytics, CelerData Cloud & Engine Internals

contact@celerdata.com (CelerData) — Mon, 02 Mar 2026 23:11:14 GMT

Our February edition is a mix of production wins, engine internals, and community deep dives—with a look at where things are heading: sub-second analytics serving both end users and AI agents at scale—from the same engine.

This month: one team shares how they rebuilt their data platform on CelerData Cloud to power both interactive dashboards and LLM-driven workloads—including their MCP server + OpenAI integration in production. Others walk through unifying fragmented Trino and ClickHouse stacks, hitting sub-second on Iceberg with materialized views, and building governed analytics with dbt. Plus a deep dive on vectorized execution, a new Snowflake Horizon integration, and community posts on federation and fast joins.

January 2026 Highlights: Roadmaps, Agentic Analytics & “How We Actually Did It”

contact@celerdata.com (CelerData) — Tue, 03 Feb 2026 19:39:18 GMT

Save your spot nowJanuary always starts with big intentions. End of January is when those intentions meet reality—so we’re here with the practical stuff: what shipped, what worked, and what’s next.

Sub-Second Analytics at Scale with Snowflake Horizon Catalog and CelerData Cloud

Chelsea Wang — Tue, 03 Feb 2026 00:25:41 GMT

Apache Iceberg is fast becoming the standard for open data, Snowflake Horizon Catalog provides the governance foundation that makes this ecosystem enterprise-ready.

By integrating CelerData Cloud with Snowflake Horizon Catalog, Snowflake customers can now enable high-performance, customer-facing analytics directly on governed Iceberg data.

Governance Meets Performance

Snowflake Horizon Catalog serves as the unified governance layer for the open ecosystem. It ensures that your Iceberg tables, whether managed by Snowflake or stored externally, are secure, compliant, and easy to manage.

CelerData Cloud, built on the open-source StarRocks project, acts as a specialized execution engine for high-demand analytical queries on top of Snowflake Horizon Catalog.

This integration allows you to run sub-second, high-concurrency analytics directly on your Snowflake-governed data. Instead of moving data around for performance, you can now leverage a single source of truth for diverse workloads, keeping your data strictly governed while enabling real-time responsiveness without data copying.

From Snowflake to StarRocks+Apache Iceberg: How Fanatics Cut 90% of Analytics Cost at 6PB Scale

Sida Shen — Mon, 05 Jan 2026 23:40:31 GMT

As digital experiences become more personalized and data-driven, many companies face mounting pressure to deliver fast, self-serve analytics—without driving up costs or adding architectural complexity. But achieving that balance is difficult, especially when historical and real-time data live in different systems.

2026 Is When Open Data, Real-Time Analytics and AI Agents Converge

Sida Shen — Thu, 25 Dec 2025 04:54:55 GMT

For years, the industry has been promised a unified data future, lakehouses built on open table formats, AI agents that operate safely on governed data, and real-time analytics embedded directly into both products and decisions. It has been a compelling narrative, but in most organizations it has remained a slide-deck ambition rather than a production reality.

December 2025 Highlights: Latest in BYOC, Performance Playbooks & Community All‑Stars

contact@celerdata.com (CelerData) — Wed, 24 Dec 2025 14:52:57 GMT

We’ve made it to the end of the year—when roadmaps get finalized, dashboards get one last performance tune-up, and everyone quietly hopes Q1 will be a little less chaotic than Q4.

November 2025 Highlights: Production Stories, Customer Wins & Community News

contact@celerdata.com (CelerData) — Mon, 01 Dec 2025 15:00:02 GMT

November feels like the perfect time to “harvest” what matters most: real outcomes from real customers. In this edition, we’re spotlighting how teams are breaking through slow dashboards, complex pipelines, and runaway costs with StarRocks and CelerData. Whether you’re running customer-facing analytics, real-time risk engines, or an open lakehouse, we hope these in-production stories spark ideas for your own architecture and make your next performance win a little easier.

Analytical Agents — New Challenges for the Underlying Data Infrastructure

Sida Shen — Tue, 25 Nov 2025 02:26:47 GMT

Analytical Agents are no longer theoretical — they are here, automating decision-making and providing instant insights with minimal human interaction in the loop. However, this rapid evolution introduces unique demands on your underlying infrastructure, pushing the requirements to unprecedented levels. Let’s explore these distinct challenges and examine some of the innovative approaches being developed to meet the rigorous demands of agent-facing analytics.

From Hours to Seconds: How Yuno Accelerated Customer-Facing Analytics By Switching Off Snowflake and Athena

Sida Shen — Fri, 21 Nov 2025 23:53:08 GMT

<1s latency | 5-second data freshness | 100s of concurrency

About Yuno

Yuno is a fast-growing fintech company that provides payment orchestration and merchant analytics across Latin America. Its platform connects merchants to hundreds of payment providers, helping them optimize routing, monitor performance, and improve conversion in real time.

With hundreds of merchants and millions of daily transactions, Yuno's business relies on delivering insights instantly — every second of delay can impact customer satisfaction and revenue.

Yuno's Customer-Facing Data Stack & Requirements

Yuno's analytics platform powers a range of customer-facing workloads that demand exceptional speed and scale, including:

Merchant dashboards for real-time approval, decline, and fraud monitoring.
Conversion funnel analytics combining checkout behavior with payment outcomes.
Routing optimization tools that help merchants dynamically adjust providers.

To support these workloads, the data platform needed to evolve into a real-time analytics layer capable of handling:

Concurrency: hundreds of simultaneous dashboard queries from merchants.
Latency: sub-second response times for interactive workloads.
Freshness: data updated within seconds of each transaction.

These requirements quickly outgrew Yuno's previous Snowflake-based architecture and drove the search for a new, high-performance analytics solution.

Previous Architecture

Before adopting CelerData, Yuno operated a hybrid data stack centered on Snowflake for analytics and Athena/Hudi for historical reporting. The system was designed for batch analytics, not for high-frequency updates or real-time dashboards.

Core Components

Snowflake – main analytical warehouse for real-time dashboards and reports.
Snowpipe + dbt – handled data ingestion and transformations.
Athena on Hudi (S3) – stored historical data and powered ad-hoc or long-tail queries.

Operational Model

Each use case or team had its own Snowflake warehouse to avoid concurrency limits.
Data was loaded in batches and merged periodically, often taking more than an hour to propagate.
Analytics teams maintained two separate query systems — Snowflake for live dashboards and Athena for deep history — resulting in duplicated logic and fragmented governance.

Challenges

This dual-system setup — Snowflake for batch analytics and Athena on Hudi for historical access — served its purpose early on but could no longer keep pace with Yuno's rapidly growing, customer-facing workloads.

Data Freshness: Data latency frequently exceeded one hour, far from the second-level goal.
Latency: Query response times could reach several seconds during peak load.
Concurrency: Snowflake's warehouse isolation limited scalability; supporting more users required provisioning new warehouses.
Scalability: As data volume grew to terabytes, with hundreds of tables and billions of rows, performance degraded and maintenance effort increased.
Data Model Complexity: The main analytics table — a large "flattened" schema — combined numerous joins from transaction and event data. Over time, this table became massive and slow to maintain, with query latency compounding as data volume grew.
Complexity & Cost: Managing two analytical systems was operationally heavy. Each required separate tuning, security, and governance, making the overall stack hard to maintain.

Yuno needed a solution that could unify real-time ingestion, fast queries, and cost efficiency.

Evaluating the Next Solution

The team evaluated multiple technologies, including ClickHouse, Apache Druid, and CelerData (powered by StarRocks).

ClickHouse offered strong query performance but lacked the lakehouse integration and native support for real-time upserts that Yuno required. Druid provided good time-series capabilities but introduced significant operational overhead and performance challenges.

CelerData stood out for several reasons:

Unified real-time and federated analytics: StarRocks is able to federate real-time data in its internal storage format and external tables on Hudi/S3 with full SQL semantics.
Native support for data upserts from CDC sources: Second-level data freshness without impacting query performance
Materialized views and shared-data architecture: Enabling high concurrency and sub-second response time even at large scale.
High Performance: CelerData's fully vectorized C++ engine and cost-based optimizer deliver consistently low-latency queries, even under high concurrency, making it ideal for customer-facing workloads

After a successful proof of concept — where CelerData handled 500 queries per second with sub-second latency and second-level freshness — Yuno decided to migrate its customer-facing workloads.

The New Architecture

Yuno's new analytics platform is built around CelerData as the central query engine, directly connected to its operational data and data lake.

Data Pipeline:

Sources: Streaming data sources through Flink CDC, preprocessed, and ingested into CelerData's primary-key tables.
Data Lake: Historical datasets remain in Hudi on S3, accessible via federated queries.
Processing: CelerData maintains both synchronous materialized views (for near-instant aggregation) and asynchronous views (for complex precomputations on large joins).
Access Layer: Merchant dashboards and internal tools query CelerData directly through APIs, achieving 5–10 seconds data freshness across transactional and aggregated data.

This architecture eliminated the need for separate batch and streaming systems. It allowed Yuno to serve both live and historical analytics from a single source of truth.

Implementation

The migration was executed in phases:

Schema Redesign: Yuno transitioned from a massive flattened table to a set of primary-key tables optimized for upserts and incremental refresh.
Materialized Views: Critical dashboards (e.g., merchant conversion rates, provider success metrics) were powered by synchronous MVs with sub-second latency. More complex metrics used asynchronous MVs that refreshed every 20–30 seconds.
Partitioning Optimization: Early ingestion issues were mitigated by switching from daily to monthly partitions, which drastically reduced IOPS and improved stability.
CI/CD and Metadata Governance: CelerData was integrated with Yuno's deployment pipeline. Every schema or column change required metadata annotations and was automatically propagated to CelerData, building a semantic layer for downstream analytics and LLM-driven query generation.
LLM and Self-Service Analytics: Yuno built an internal layer where users could type natural-language questions, which the LLM converted to SQL queries on CelerData. The system reused metadata and sample data to generate charts and dashboards automatically, turning complex analytics into a self-service experience.
Observability and Troubleshooting: The team enhanced ingestion reliability and visibility, adding monitoring for partition health, versioning, and MV refresh status to prevent previous stability issues.

Results

After migrating to CelerData, Yuno achieved measurable improvements:

Query Latency: Reduced from ~3 seconds on Snowflake to under 1 second for most dashboards.
Data Freshness: Improved from 1 hour to around 5 seconds for core transactional data.
Concurrency: Supported hundreds of concurrent dashboard queries without performance degradation.
Cost Efficiency: Eliminated the need for multiple Snowflake warehouses and reduced compute cost per query by over 40%.
Reliability: The shared-data architecture and optimized partitioning significantly improved ingestion stability.

Operational teams can now monitor provider performance, identify conversion drops, and adjust routing rules in real time — transforming analytics from a retrospective tool into a live decision engine.

Future Work

Yuno continues to evolve its analytics platform with CelerData. The next steps include:

Integrating S3 Express One Zone for lower latency and reduced cost of hot data storage.
Enhancing LLM-driven analytics by expanding the internal knowledge base with query examples and column semantics.
Improving disaster recovery and metadata resilience to further harden ingestion during upgrades and reboots.
Expanding customer-facing analytics to new regions and product lines, supported by CelerData's scalability and lakehouse integration.

With CelerData as the foundation, Yuno has built a modern analytics architecture that delivers real-time insights at scale — combining the speed of a database with the openness of a lakehouse, and turning data into a true competitive advantage.