What’s Next for the Analytics Engineer? - Reflections From Coalesce 2022

Phew! I’m not the only one who has spent a significant chunk of my analytics career stuck in the world of tech debt. Talks like these are important because they give everyone a glimpse into the side of analytics engineering that is not super glamorous, but a foundational skill that you absolutely have to build.

I Wish I Was at Coalesce for This

• Babies and bathwater: Is Kimball still relevant? (Sydney Burns & Josh Devlin)

Data modeling technique has become an “agree to disagree” situation among the analytics engineering crowd. Kimball’s method dates back to 1996 so it doesn’t feel like a natural fit in the modern data stack. Or does it? This talk explained why dimensional modeling is still relevant today–well, at least until something better comes along to replace it.

Most Likely to Recommend to Non-Data Practioners

• How not to be a terrible writer (Justin Gage)

The topic of writing is near and dear to my heart. As data practitioners, I believe we can learn a lot about how to deliver compelling analytics when we consider the art and process behind storytelling. This talk had some really substantial writing advice that can apply far beyond the scope of analytics and the audience of Coalesce.

My Own Journey, Post-Analytics Engineer

My Own Journey, Post-Analytics Engineer

I became an analytics engineer in 2019. Since then, the role has exploded, both in numbers and demand, which we saw in the massive turnout at Coalesce last week. Reflecting on my job experiences though, I can’t quite reconcile the enthusiasm I witnessed at the conference with my own day-to-day responsibilities working as an analytics engineer.

For me, the role felt too tightly scoped to using dbt to prepare data for analytics, without having any insight into whether my work created real value. To expand on the tasks I was assigned, I had to reach into an area that already belonged to a different role on the team.

In an established organization, most teams agree that the data warehouse administration and pipelines belong to data engineers, and that data analysts own the BI tools and collaborate with the business stakeholders to define key metrics.

The analytics engineer lies somewhere in the middle, focusing on the “T” of the ELT (extract, load, transform) process. This includes shaping the data and resolving inconsistencies. In my experience, transformation work can become repetitive and can feel like a somewhat low-profile task without reach outside of the data team. While the work is important, there’s not a lot of room for skill development once you achieve competence with dbt.

I never saw the role as a permanent fit but rather an entry point into working in the modern data stack. So while the data community gathered in New Orleans, I began a new journey at Hightouch, as a data advocate. I’m excited about the advocate role because I’ll be able to draw from my previous engineering background (from the pre-modern data stack days) and explore modern data team challenges with a broader and more altruistic perspective, rather than being limited to one organization’s specific use case.

Today's Analytics Engineer

Today's Analytics Engineer

Analytics engineering has become a staple of most data teams, somehow inextricably linked to our use of dbt as a product. There’s a lot of hype around the job, but much of that hype comes from outside the role itself. Within the data community, a small portion of the thought leadership that is talking about analytics engineering actually holds the title of analytics engineer.

Analytics engineering is still a new and relatively accessible discipline. Most companies expect a baseline of SQL experience to become an entry-level analytics engineer, with the expectation that a person will learn dbt on the job. Some job postings also mention data modeling, but there’s no widespread consensus about the right technique for modeling data within cloud data warehouses. Recently I’ve observed that the use of dbt and the technique of data modeling have become somewhat interchangeable, as though being the product of a dbt run inherently makes something a model. This is an area I’m eager to explore from an advocate’s perspective.

The day-to-day tasks you can expect when you become an analytics engineer are hard to classify because so much depends on the organization and team. Some analytics engineers are tasked with implementing a new project from the ground up while others are tasked with unraveling tech debt from an inventory of spreadsheets and messy queries. Both use dbt, but it’s not really the same job at all.

Looking to the Future

Looking to the Future

There are a lot of resources that describe how to become an analytics engineer and how to advance your dbt skills, but not as many about how to navigate your career once you become an analytics engineer. It’s an area I expect to expand post-Coalesce, as more analytics engineers increase their tenure in the role and as more and more data practitioners migrate into the modern data stack.

There are many possibilities for the evolution of the role. My guess is that we will see different variations of analytics engineers as the people doing the job shape the future. Because of my own path, I can see analytics engineers moving into either a more technical role, like a data engineer, or into a leadership role with more influence. I can also imagine the scope of the analytics engineer growing as we further refine data modeling practices for cloud warehouses and expand the ways we use our data models to drive business action.

In my new role as a data advocate at Hightouch, I’m excited to delve into these topics with the community and plan to share my learnings, resources, and tips along the way.