The Apache Iceberg project introduced an open table storage format to store structured data in object storage systems (e.g., AWS S3, MinIO) for analytical purposes. Much like the S3 protocol became the de facto standard for accessing and managing object storage, Iceberg’s catalog API has the potential to
In this post, I will argue that today’s query engines, such as Apache Spark, do not effectively interact with lake houses due to limitations in handling object storage semantics, lack of integration with catalog systems like Iceberg for governance and lineage, and inadequate support for lakehouse-specific features. I believe
When working with collections in Rust programming language, especially arrays or vectors, it's common to encounter elements inside arrays wrapped in Option or Result types. Rust provides powerful iterator methods like map and filter_map to manipulate these arrays efficiently. In this blog post, we'll explore
In Rust programming language, lifetimes are a powerful feature that ensures memory safety without needing a garbage collector. However, they can sometimes introduce complexity, mainly when dealing with borrowed references in structs and functions. This complexity can spread throughout your codebase, making it harder to read and maintain. This post
The rapid advancement of large language models (LLMs) has transformed the landscape of artificial intelligence. As these models grow in size and complexity, efficient distributed training and inference have become critical challenges. Apache Ray, a popular distributed computing framework, has gained attention for its flexibility and ease of use. However,
This is hackintoshrao, a brand new site by Karthic Rao that's just getting started. Things will be up and running here shortly, but you can subscribe in the meantime if you'd like to stay up to date and receive emails when new content is published!