How an Analytics Engineer Uses AI Daily
Artificial Intelligence (AI) has become a game-changer in many industries, and data analytics is no exception. AI tools help automate repetitive tasks, improve efficiency, and allow professionals to focus on higher-value work. In the field of analytics engineering, where coding, data modeling, and pipeline management are critical, AI can mean the difference between spending hours on tedious tasks and delivering insights quickly.
In this post, I’ll talk about two AI tools I often use to streamline my analytics engineering workflow: GitHub Copilot and the dbt Power User extension in VS Code. Then I’ll cover dbt Labs’ upcoming AI tool for dbt Cloud users, called dbt Copilot.
GitHub Copilot
GitHub Copilot has been my virtual buddy this year, suggesting code snippets, clarifying syntax, and even completing entire lines or functions. For example, this year I worked on a customized ingestion solution written in Python by a senior data engineer. GitHub Copilot helped me by explaining code and boosting my confidence when its recommendations aligned with my coding approach. While not all the suggestions have been perfect, having a strong foundation and working understanding of programming helps me evaluate them. What’s cool is that Copilot frequently predicts my next steps, saving time on troubleshooting and reducing common errors from the start.
dbt Power User
The dbt Power User extension in VS Code has made my dbt Core development feel like I’m actually on dbt Cloud by offering features such as UI buttons where I can build, run, or test models with one click. There is also instant access to documentation allowing me to see model lineage without invoking “dbt docs generate” and “dbt docs serve” first, which saves time during development and improves my developer experience.
A major time-saver made possible by its bundled AI functionality is a feature where I can generate documentation for an entire model or a single column with one click. The extension’s Founder Engineer, Michiel De Smet, and Anton Goncharuk from HubSpot gave a talk at Coalesce 2024 demonstrating yet more features to come that I’m excited about. One of those is the “notebooks” feature that looks like a Jupyter notebook, where you can combine dbt model code, Jinja, and Python. You can refer to data returned by dbt SQL in Python, do some analysis, or troubleshoot a problem in a model, and share the notebook with others in the team. I’m also excited about other features of this extension like auto-generating dbt models from plain SQL, auto-generating tests, and project governance checks.
dbt Copilot
At Coalesce 2024, Drew Banin at dbt Labs announced dbt Copilot, an AI agent directly integrated inside dbt Cloud. This feature will be great for users who already use dbt Cloud. I’m especially interested in trying out its data tests and documentation generation features and comparing them to what I’ll get from the dbt Power User extension. I’m anticipating that dbt Copilot will reduce even more of the effort for standing up greenfield data projects (something we do a lot at Brooklyn Data) and offer guidance for optimizing complex or expensive data transformations (also one of our specialties!).
How These AI Tools Will Transform Analytics Engineering
Looking ahead, AI tools like GitHub Copilot, the dbt Power User extension, and dbt Copilot will transform how we approach analytics engineering. Are they going to take my job? 🤔 I don’t think so!
Instead, as these tools become more intuitive, they’ll likely help us automate the easy stuff earlier in projects, optimize our models, and let us solve higher-level challenges as the data transformation layer grows and supports more teams within the business. We can work quicker and smarter, explore new techniques, and deliver insights faster incorporating AI into our daily workflow.
Want to know more about how GitHub Copilot, dbt Power User, and dbt Copilot can streamline your analytics workflows? Reach out. Our experts can tell you more about how they can save your analytics engineers valuable time.