Summary
Spark lineage is an important yet often overlooked aspect of data engineering. This article outlines the development of a solution for building Spark lineage in data lakes, enabling professionals to gain better insights into data flows and dependencies.
Deepen your knowledge
Knowledge Base
BI Implementation Roadmap — From Vision to Working Dashboard
Practical BI implementation roadmap: from strategy and data inventory to dashboards and adoption. Avoid common pitfalls ...
Knowledge BaseData-Driven Work — How to get started as an organization
Learn how to become a data-driven organization. From data maturity to culture change: a practical step-by-step guide wit...
Knowledge BaseData Engineer vs Data Analyst: what's the difference?
Discover the difference between a Data Engineer and Data Analyst: tasks, tools, salary and career paths. Which role suit...
Knowledge BaseData Governance for SMBs — A practical approach
What is data governance and how do you approach it as an SMB? A practical guide covering GDPR compliance, data quality, ...
Knowledge BaseData Lakehouse Explained — The best of both worlds
What is a data lakehouse and why does it combine the best of data warehouses and data lakes? Architecture, comparison, a...
Knowledge BaseETL Explained — Extract, Transform, Load in plain language
What is ETL? Learn how Extract, Transform, and Load works, the difference with ELT, and which tools to use. Clearly expl...
Knowledge BaseWhat is Business Intelligence? Definition, examples and tools
What is business intelligence (BI)? Learn about the definition, BI stack, real-world examples, popular tools, and 2026 t...
