Menu

Post image 1
Post image 2
Post image 3
1 / 3
0

Building a PSX Data Pipeline: How I get introduced to dbt

DEV Community·muzzamilanis·about 1 month ago
#6FNWlgqP
Reading 0:00
15s threshold

I've been in software development since 2017. Started as a .NET developer, transitioned into SDE-III, and for the past 4 years working as data engineer. Eight years in tech means I'm not new to building things but data engineering was a different world for me. More of a sql related task but different in its own uniqueness. Like a different world in itself. I knew dbt existed. I'd seen it in job descriptions. But I had no real idea what it actually did or why companies were treating it like a big deal. My mental model was basically: "May be a new tool like most other available in market." Spoiler: I was wrong. Here's what I built and what changed my mind. The Problem I Was Trying to Solve I wanted a real data engineering project I could point to. Not a tutorial I followed on YouTube. Not a dataset from Kaggle. Something with actual data, actual decisions, and actual engineering challenges. I'm based in Pakistan and the Pakistan Stock Exchange (PSX) had no decent open data tooling.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More