dbt Core™ is an open-source tool by dbt Labs that enables the application of software engineering principles to data analytics. Simply put, it allows you to write, test, document and version control your SQL code in a single repository. No more fragmented, undocumented and untested SQL code (views) in your database!
So what are the tangible benefits that the adoption of dbt can provide for your team?
First and foremost, dbt provides a standardized way of organizing and testing your analytical SQL models in a single source-of-truth repository. This is especially important if your data warehouse has grown into a spaghetti mess of untested SQL views. Having tests defined for your analytical models and being able to run the tests on-demand will give your team the confidence to make changes to existing models and deploy them to production. This in turn increases your team’s productivity.
Second, you can automate tests in dbt Cloud, CI/CD, Airflow or any other pipeline orchestrator and get notified when your models break (e.g., due to source data changes). This gives you a head start on fixing the defects before they ever reach your end users.
Third, you can generate and publish a documentation site for your data warehouse using dbt’s built-in “dbt docs” feature. The documentation site allows anyone in your organization to explore the analytical SQL models and their relationships in your data warehouse. This is a step towards having documented and curated data sets available for anyone in your organization to experiment with.
I’m a dbt Preferred Consulting Provider and I provide dbt deployment, development and training (English and Finnish) services for organizations that have an existing data warehouse or other relational SQL storage, or do not yet have a data warehouse but are planning on setting one up.
Being able to adopt dbt in your organization doesn’t require “hardcore” data engineering skills. I’ve trained dbt to a wide range of audiences ranging from product development and mechanical engineers to data analysts and scientists. As a prerequisite for the training you only need to know SQL and some basics of git version control. And if those are not in your toolbox yet, we can cover them during the training.
Data engineers are the backend engineers of the data world. We take care of the data infrastructure so that data analysts and scientists can focus on what they do best.
I provide data infrastructure review, design and implementation services. In development work I adhere to software engineering and DevOps best practices, such as automated (end-to-end) testing, CI/CD and Infra-as-Code. I’m also a proponent of the Test-Driven-Development (TDD) methodology.
Analytics engineer is an emerging role in the data world. The role is similar to that of a data engineer but it leans more towards data analytics, as the name might suggest. In other words, analytics engineers provide clean data to data analysts using automated processes.
I can help your organization with your analytics needs ranging from simple scripts to more scalable and automated solutions. For more information on my analytics engineering portfolio, see dbt deployment and training.
We all have our strengths – and that’s a good thing! But sometimes you need specific skills and experience to take things to the next level. Let me help your team get there.
Kind words from clients.
“We used to have most of our analytics in Excels which eventually became a bottleneck for our product development. Simo helped us automate these processes by designing and implementing a data infrastructure tailor-made for our needs. Furthermore, he trained our product development team with no previous data processing experience so that they could understand and independently make use of our new analytics capabilities. Now we can handle large amounts of data in an automated manner which has accelerated our product development efforts and improved our decision making. We are really happy with the services of data&mie and will definitely continue our collaboration!”
International Advanced Technologies at Globus Medical
“Over the past three years, as our company scaled rapidly – so did the demand for rapid data driven insights from all departments. While modern ETL tools, Data Warehouses & Analytics platforms allowed us to keep pace with this increased demand – we quickly hit scalability issues with undocumented / siloed schemas and SQL models and realized we really need to correct course.
Not knowing where to start, we connected with Simo via the dbt community – and were immediately impressed by his knowledge of the dbt framework. Simo not only took on our initial project, he continuously trained our team and over the course of a couple of months helped our team mature from a free for all SQL shop to one with a structured dbt workflow. Simo continues to help us by directly developing some of our most complex data models with a keen attention to accuracy and detail, but also by sharing best practices and performing peer reviews with the goal of getting the team to be self reliant. This honest, education driven approach really sets Simo apart from other consultants I’ve worked with.”
Senior Director of Analytics, Insights & MIS