DataGalaxy Tech Summit
NYC 2024

For decades, data modeling has been fragmented by use cases, such as applications, analytics, machine learning, and AI. This leads to data siloing and “throwing data over the wall.” With the emergence of AI, streaming data, and “shifting left,” data modeling is changing. These siloed approaches are insufficient for the diverse world of data use cases. Today’s practitioners must possess an end-to-end understanding of the myriad techniques for modeling data throughout the data lifecycle. This presentation covers “mixed model arts,” which advocates converging various data modeling methods and innovating new ones.

How did we get to the point that a single machine can compete with distributed systems? Where does DuckDB shine? What makes for a good pilot project in your organization? Can you really replace your warehouse with DuckDB somehow? Discover how DuckDB opens up many creative possibilities for data engineers by putting a column-store database engine in a tiny library you put anywhere.

Redpanda is a new Kafka known for its operationally simple, developer-friendly approach. Recently, Redpanda has been making significant changes to its platform with capabilities such as native Topic to Iceberg integration, flexible topic configuration for performance vs. cost, and recently acquired Benthos, which gets immediate upgrades with WASM and GPU / AI integrations. In this talk, we’ll get an update from Redpanda on some of the details behind these new features and an inside look at what else is on the roadmap. You will also have a chance to ask questions and get some Redpanda swag!

Data modeling is important. but first, you need to make sure you’re working on the right things.

In this talk, we’ll explore how modern data professionals are often unaware of the lessons from past data architecture mistakes—leading to inefficiencies and avoidable errors. We’ll journey through the evolution of data models, from the simplicity of the star schema to the complexity of the snowflake schema, and examine how today’s Medallion Architecture offers a fresh, more agile approach compared to traditional data warehousing methods.

By embracing tried-and-true principles while adopting new innovations, we can create data architectures that are not only efficient but scalable and adaptable to future challenges.