Apache Spark Scala Library Development with Databricks
The movie Toy Story was released in 1995 by Pixar as the first feature-length computer animated film. Even though the animators had professional workstations to work with, they started sketching out...
View ArticleManaging and Securing Credentials in Databricks for Apache Spark Jobs
Since Apache Spark separates compute from storage, every Spark Job requires a set of credentials to connect to disparate data sources. Storing those credentials in the clear can be a security risk if...
View ArticleBuilding Complex Data Pipelines with Unified Analytics Platform
Introduction Big data practitioners often post recurring questions on Quora: What is data engineering? How to become a data scientist? What’s a data analyst? Apart from understanding these roles and...
View ArticleTop 5 Reasons to Convert Your Cloud Data Lake to a Delta Lake
If you examine the agenda for any of the Spark Summits in the past five years, you will notice that there is no shortage of talks on how best to architect a data lake in the cloud using Apache Spark™...
View ArticleClik here to view.
Upgrade Production Workloads to Be Safer, Easier, and Faster With Databricks...
What a difference a year makes. One year ago, Databricks Runtime version (DBR) 6.4 was released — followed by 8 more DBR releases. But now it’s time to plan for an upgrade to 7.3 for Long-Term Support...
View ArticleClik here to view.
Building a Cybersecurity Lakehouse for CrowdStrike Falcon Events
Endpoint data is required by security teams for threat detection, threat hunting, incident investigations and to meet compliance requirements. The data volumes can be terabytes per day or petabytes per...
View ArticleClik here to view.
How Databricks’ Data Team Built a Lakehouse Across 3 Clouds and 50+ Regions
The internal logging infrastructure at Databricks has evolved over the years and we have learned a few lessons along the way about how to maintain a highly available log pipeline across multiple clouds...
View Article
